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Chapter  One:  Introduction 


This  Software  Enabled  Control  project,  sponsored  by  DARPA,  was  a  landmark 
project  at  Berkeley  in  several  different  ways.  It  supported  the  research  of  three 
faculty  investigators:  Shankar  Sastry,  Tom  Henzinger  and  Edward  Lee  and 
several  graduate  students  (by  our  estimate  over  30  graduate  students  over  the 
life  of  the  project)  and  several  post  doctoral  scholars.  Some  examples  of  the 
project  alumni  and  their  current  positions  are 

•  George  Pappas,  University  of  Pennsylvania 

•  Rene  Vidal,  Johns  Hopkins  University 

•  Jin  Kim,  Seoul  National  University 

•  Omid  Shakernia,  Northrop  Grumman 

•  Judy  Liebmann,  Lawrence  Livermore  National  Laboratories 

•  Benjamin  Horowitz,  Lawrence  Livermore  National  Laboratory 

•  Cedric  Ma,  Northrop  Grumman 

•  John  Koo,  Vanderbilt  University 

•  Maria  Prandini,  University  of  Milano,  Italy 

•  Karl  Johann  Johansson,  Royal  Institute  of  Technology,  Stockholm, 
Sweden 

•  Christoph  Kirsch  Meyer,  University  of  Salzburg,  Austria 

•  Steven  Neundorffer,  Xilinx  Corporation 

•  Luca  De  Alfaro,  University  of  California,  Santa  Cruz 

•  Rupak  Majumdar,  University  of  California-  Los  Angeles 

•  Jie  Liu,  Microsoft  Research  Corporation 

•  Lara  Crawford,  PARC  Research  Center,  Palo  Alto 

Major  Research  in  the  project  fell  into  three  areas: 

Hybrid  and  Multi-Modal  Systems 

The  full  theory  of  hybrid  systems  and  safe  design  of  controllers  have  been 
developed  for  safety  and  liveness  specifications  for  hybrid  systems.  Work  has 
been  pioneered  on  both  deterministic  and  stochastic  hybrid  control  systems  and 
developed  analysis  techniques.  An  annual  workshop  called  "Hybrid  Systems: 
Computation  and  Control"  has  been  started,  which  is  now  the  leading  workshop 
on  the  topic  in  the  world. 

Design  of  Embedded  Systems  and  High  Confidence  Systems 

We  developed  time  triggered  and  asynchronous  techniques  for  the  synthesis  of 
hybrid  systems.  We  developed  tools  for  use  with  Ptolemy  such  as  HyVisual  and 
other  tools  such  as  Giotto  and  Massacio  for  designing  time  triggered  systems. 
We  began  a  workshop  series  called  Embedded  Software  (Emsoft)  which  has 
become  the  leading  workshop  in  the  world  for  the  topic 
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Experimental  Work 

We  demonstrated  on  our  helicopter  platform  during  the  course  of  the  project 
numerous  innovative  projects  using  the  methods  developed  during  the  course  of 
our  research: 

Landing  on  pitching  decks 

Reinforcement  Learning  based  control 

Acrobatic  Maneuvers  using  reinforcement  learning 

Conflict  Detection  and  Resolution 

Formation  Flying 

Air  to  Air  Combat 

Map  Building  and  obstacle  avoidance 

A  number  of  these  approaches  were  transitioned  to  other  DARPA  programs  such 
as  UCAR,  the  Perch  and  Move  program,  the  A-160  Maverick  transition  program. 
Additionally  our  experimental  work  enabled  us  to  transition  the  results  to  the 
Boeing  Open  Control  Platform  for  the  Capstone  Demo  in  August  2004. 

Accomplishments  from  1999-2004 

We  considered  the  design  problem  of  embedded  software  for  multi-vehicle  multi¬ 
modal  systems.  Motivated  by  our  design  experience  on  the  development  of 
embedded  software  for  our  helicopter-based  unmanned  aerial  vehicles  which  are 
composed  of  heterogeneous  components,  we  believe  that  at  the  level  closest  to 
the  environment  under  control,  the  embedded  software  needs  to  be  time- 
triggered  for  guaranteed  safety;  at  the  higher  levels,  we  advocate  asynchronous 
hybrid  controller  design.  Since  the  simulation  is  a  precursor  to  the 
implementation  of  the  control  laws  on  real  application.  We  will  use  a 
heterogeneous  model  simulator  for  high-level  control  designs.  This  will  be  done 
in  a  way  consistent  with  and  in  support  of  the  DARPA  Open  Control  Platform 
(OCP),  in  order  to  bridge  the  gap  between  functional  control  design  and  software 
implementation.  Work  focused  on  the  realization  of  the  system  for  hardware-in- 
the-loop  (HIL)  simulation.  HIL  simulation  allows  testing  an  embedded  computing 
device  by  progressively  replacing  the  real-world  environment  at  the  device's 
input-output  interface  with  a  simulated  environment.  HIL  simulation  facilitates 
repeatable  testing,  as  well  as  quantification  of  design  tradeoffs.  We  began  the 
construction  of  a  HIL  simulation  system  for  a  group  of  unmanned  aerial  vehicles 
(UAVs).  We  also  began  planning  for  mid-term  demonstration  is  to  fly  hybrid 
control  software  on  the  Berkeley  helicopters.  Then,  in  the  final  demonstration,  we 
planned  to  use  the  OCP  publish-and-subscribe  architecture  for  mutli-vehicle 
coordination.  By  way  of  abbreviations,  MMCC  stands  for  MultiModal  Cooperative 
Control,  HCSS  is  High  Confidence  Software  and  Systems. 
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Chapter  Two:  Accomplishments  in  2000 


We  extended  the  theory  and  implementation  of  the  model  checker  HyTech  to 
cope  with  rectangular  and  nonlinear  hybrid  systems.  We  also  developed 
asystematic  classification  of  which  requirements  of  hybrid  systems  can  be  model 
checked.  Finally,  we  started  a  new  project,  FRESCO,  on  faithfully  implementing 
hybrid  models  in  real-time  software. 

Beyond  HyTech:  Hybrid  Systems  Analysis  Using  Interval  Numerical 
Methods 

Since  hybrid  embedded  systems  are  pervasive  and  often  safety-critical, 
guarantees  about  their  correct  performance  are  desirable.  The  hybrid  systems 
model  checker  HyTech  provides  such  guarantees  and  has  successfully  verified 
some  systems.  However,  HyTech  severely  restricts  the  continuous  dynamics  of 
the  system  being  analyzed  and,  therefore,  often  forces  the  use  of  prohibitively 
expensive  discrete  and  polyhedral  abstractions.  Ben  Horowitz,  Rupak 
Majumdar,  and  Howard  Wong-Toi  designed  a  new  algorithm,  which  is  capable  of 
directly  verifying  hybrid  systems  with  general  continuous  dynamics,  such  as 
linear  and  nonlinear  differential  equations.  The  new  algorithm  conservatively 
over-approximates  the  reachable  states  of  a  hybrid  automaton  by  using  interval 
numerical  methods.  Interval  numerical  methods  return  sets  of  points  that 
enclose  the  true  result  of  numerical  computation  and,  thus,  avoid  distortions  due 
to  the  accumulation  of  round-off  errors.  The  new  algorithm  is  implemented  in  a 
successor  tool  to  HyTech  called  HyperTech.  Three  examples  are  considered:  a 
thermostat  with  delay,  a  two-tank  water  system,  and  an  air-traffic  collision 
avoidance  protocol.  HyperTech  enables  the  direct,  fully  automatic  analysis  of 
these  systems, which  is  also  more  accurate  than  the  use  of  polyhedral 
abstractions. 


Robust  Undecidability  of  Timed  and  Hybrid  Systems 

The  algorithmic  approach  to  the  analysis  of  timed  and  hybrid  systems  is 
fundamentally  limited  by  undecidability,  of  universality  in  the  timed  case  (where 
all  continuous  variables  are  clocks),  and  of  emptiness  in  the  rectangular  case 
(which  includes  drifting  clocks).  Traditional  proofs  of  undecidability  encode  a 
single  Turing  computation  by  a  single  timed  trajectory.  These  proofs  have 
nurtured  the  hope  that  the  introduction  of  "fuzziness"  into  timed  and  hybrid 
models  (in  the  sense  that  a  system  cannot  distinguish  between  trajectories  that 
are  sufficiently  similar)  may  lead  to  decidability.  J-F  Raskin  showed  that  this  is 
not  the  case,  by  sharpening  both  fundamental  undecidability  results.  Besides  the 
obvious  blow  our  results  deal  to  the  algorithmic  method,  they  also  prove  that  the 
standard  model  of  timed  and  hybrid  systems,  while  not  "robust"  in  its  definition  of 
trajectory  acceptance  (which  is  affected  by  tiny  perturbations  in  the  timing 
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of  events),  is  quite  robust  in  its  mathematical  properties:  the  undecidability 
barriers  are  not  affected  by  reasonable  perturbations  of  the  model. 


Symbolic  Model  Checking  for  Rectangular  Hybrid  Systems 

An  important  case  of  hybrid  systems  are  the  rectangular  automata.  First, 
rectangular  dynamics  can  naturally  and  arbitrarily  closely  approximate  more 
general,  nonlinear  dynamics.  Second,  rectangular  automata  are  the  most 
general  type  of  hybrid  systems  for  which  model  checking  —in  particular,  LTL 
model  checking-  is  decidable.  However,  on  one  hand,  the  original  proofs  of 
decidability  did  not  suggest  practical  algorithms  and,  on  the  other  hand,  practical 
symbolic  model-checking  procedures  -such  as  those  implemented  in  HyTech- 
were  not  known  to  terminate  on  rectangular  automata.  Rupak  Majumdar 
remedied  this  unsatisfactory  situation:  he  developed  a  symbolic  method  for  LTL 
model  checking  which  can  be  performed  by  HyTech  and  is  guaranteed  to 
terminate  on  all  rectangular  automata.  This  is  done  by  proving  that  our  method 
for  symbolic  LTL  model  checking  terminates  on  an  infinite-state  transition  system 
if  the  trace-equivalence  relation  of  the  system  has  finite  index,  which  is  the  case 
for  all  rectangular  automata. 


A  Classification  of  Symbolic  Transition  Systems 

Rupak  Majumdar  defined  five  increasingly  comprehensive  classes  of  infinite- 
state  systems,  called  STS  1-5,  whose  state  spaces  have  finitary  structure.  For 
four  of  these  classes,  examples  from  hybrid  systems  can  be  provided. 

STS1:  These  are  the  systems  with  finite  bi-similarity  quotients.  They  can  be 
analyzed  symbolically  by  (1)  iterating  the  predecessor  and  Boolean  operations 
starting  from  a  finite  set  of  observable  state  sets,  and  (2)  terminating  when  no 
new  state  sets  are  generated.  This  enables  model  checking  of  the  mu-calculus. 

STS2:  These  are  the  systems  with  finite  similarity  quotients.  They  can  be 
analyzed  symbolically  by  iterating  the  predecessor  and  positive  Boolean 
operations.  This  enables  model  checking  of  the  existential  and  universal 
fragments  of  the  mu-calculus. 

STS3:  These  are  the  systems  with  finite  trace-equivalence  quotients.  They  can 
be  analyzed  symbolically  by  iterating  the  predecessor  operation  and  a  restricted 
form  of  positive  Boolean  operations  (intersection  is  restricted  to  intersection  with 
observables).  This  enables  model  checking  of  linear  temporal  logic. 

STS4:  These  are  the  systems  with  finite  distance-equivalence  quotients  (two 
states  are  equivalent  if  for  every  distance  d,  the  same  observables  can  be 
reached  in  d  transitions).  The  systems  in  this  class  can  be  analyzed  symbolically 
by  iterating  the  predecessor  operation  and  terminating  when  no  new  state  sets 
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are  generated.  This  enables  model  checking  of  the  existential  conjunction-free 
and  universal  disjunction-free  fragments  of  the  mu-calculus. 

STS5:  These  are  the  systems  with  finite  bounded-reachability  quotients  (two 
states  are  equivalent  if  for  every  distance  d,  the  same  observables  can  be 
reached  in  d  or  fewer  transitions).  The  systems  in  this  class  can  be  analyzed 
symbolically  by  iterating  the  predecessor  operation  and  terminating  when  no  new 
states  are  encountered.  This  enables  model  checking  of  reachability  properties. 


FRESCO:  Formal  Real-time  Software  Components 

We  have  been  designing  a  formal,  structured  modeling  language  for  hybrid 
control  systems,  called  Masaccio,  and  a  time-triggered  implementation  language, 
called  Giotto.  Our  eventual  goal  is  to  establish  requirements  in  Masaccio  by 
model  checking,  in  a  way  which  guarantees  that  the  Giotto  implementation  meets 
the  requirements. 

Embedded  Software  Design 

We  have  been  experimenting  extensively  with  embedded  computing  devices, 
sensors,  and  actuators,  to  gain  experience  and  to  identify  a  suite  of  experimental 
platforms  that  are  safer  to  experiment  with  than  the  Bear  Project  helicopters.  The 
objective  is  to  develop  techniques  for  portable,  component-based  design  of 
embedded  software.  We  plan  to  use  the  same  techniques  for  programming  a 
variety  of  platforms,  including,  eventually,  the  autonomous  helicopters.  We  have 
performed  quite  a  few  investigations,  and  made  quite  a  bit  of  progress  in 
understanding  this  space. 

Jini  and  Java  Spaces 

To  give  us  a  versatile  experimental  platform  in  anticipation  of  the  Boeing  OCP, 
we  developed  a  publish-and-subscribe  mechanism  in  Ptolemy  II  based  on  Java 
Spaces,  a  Linda-style  distributed  messaging  system  from  Sun.  Jie  Liu  and 
Yuhong  Xiong  created  a  demonstration  where  a  publisher  extracts  stock  market 
data  from  a  web  source  and  and  subscriber  consumes  that  data  to  generate 
trend  plots.  This  demonstration  shows  the  network  integration  and  publish-and- 
subscribe  interactions  working  with  Ptolemy  II.  Moreover,  the  demo  used  Jini,  a 
networked  service-discovery  mechanism  from  Sun,  intended  for  embedded 
devices.  Service  discovery  is  not  currently  part  of  the  OCP  plan,  but  we  hope  to 
develop  a  better  understanding  of  its  role  to  help  determine  whether  it  should  be 
incorporated  later,  and  in  what  form. 

Pioneer  and  Jini 

Jie  Liu  and  Steve  Neuendorffer  created  a  software  infrastructure  in  Java  for 
programming  a  Pioneer  mobile  robot  borrowed  from  the  Bear  project  and  the 
Legos  using  the  same  code.  This  is  our  first  serious  demonstration  of  platform 
independent  software  development.  The  Pioneer  is  a  much  more  substantial 
platform  than  the  Legos,  and  has  a  single-board  computer  running  Linux.  Their 
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software  uses  Jini  to  distribute  code  to  robots,  or  in  the  case  of  the  Legos,  to  a 
proxy  for  the  robots  running  on  a  laptop  computer.  The  code  that  is  distributed  is 
Java  bytecode.  When  a  task  is  needed  of  a  robot,  an  instance  of  a  Java  class  is 
posted  to  the  Jini  server,  and  the  robots  are  notified.  Any  robot  that  is  available 
and  implements  an  interface  compatible  with  the  posted  task  can  grab  the  code 
from  the  Jini  server  to  execute  the  task. 

Robot  Arm 

Chamberlain  Fong  assembled  from  a  kit  a  small  robot  arm  with  four  degree  of 
freedom  motion  and  a  gripper.  He  built  a  demonstration  of  a  controller  which  this 
arm  that  uses  a  new  domain  in  Ptolemy  II  that  he  is  developing,  DT  (discrete 
time).  The  demonstration  includes  an  on  screen  animation,  through  which 
commands  are  entered  using  the  keyboard,  and  the  mouse  is  used  for  changing 
the  viewpoint  of  the  arm  simulator.  The  communication  between  software 
components  is  mostly  angles  and  polygons  generated  by  a  forward  kinematics 
controller.  Chamberlain  is  working  on  an  inverse  kinematics  controller.  The  robot 
arm  is  connected  to  a  host  computer  through  the  serial  port. 

Handspring  Visors 

Christopher  Hylands  experimented  with  the  KVM  (a  minimal  Java  virtual 
machine)  running  on  Handspring  Visors  (handheld  electronic  organizers).  We 
selected  the  Handspring  Visor  instead  of  the  more  popular  Palm  Pilot  because 
the  Visor  includes  a  bus  with  a  published  interface,  and  thus  has  much  more 
promise  for  use  in  embedded  systems.  Christopher  was  able  to  get  a  small 
portion  of  the  Ptolemy  II  code  to  compile  for  the  KVM,  and  was  able  to  port  a 
small  process  networks  application.  Christopher  attempted  to  get  the  IR  port  on 
the  Handspring  to  communicate  with  the  Legos,  but  was  unsuccessful. 

Dallas  Tini 

We  obtained  a  pair  of  Dallas  Seminconductor  Tini  boards,  small  single-board 
computers  that  include  a  Java  virtual  machine  in  a  DIMM  form  factor.  They 
support  a  serial  port,  a  1-wire  interface,  CAN  bus  and  lObaseT  ethernet. 

http://www.ibutton.com/TINI/software/index.html  says: 

"The  Java  VM  on  TINI  conforms  to  Sun's  *Embedded*  Java  platform,  version 
1.1  of  the  Java  API.  Embedded  in  TINI's  flash  memory  are  java.lang,  java.net, 
java.io,  java. util;  there  is  room  in  RAM  for  other  packages  of  your  choice.  Also  in 
flash  memory  are  com.dalsemi  packages  for  accessing  the  TINI  command  shell 
slush,  the  1-Wire®  bus  and  several  system  parameters." 

TINI  implements  none  of  the  serialization,  reflection,  and  dynamic  class  loader, 
so  it  is  still  far  from  RMI.  That  is,  at  this  time,  it  is  impossible  to  make  the  TINI 
board  a  Jini  device  and  access  the  Java  Space  directly.  It  is  doubtful  that  these 
devices  would  be  easy  to  interface  to  the  OCP,  except  through  proxies  running 
on  a  more  substantial  host.  Nonetheless,  there  are  interesting  possibilities. 
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Dallas  makes  a  small  memory  device  called  an  iButton.  The  idea  is  that  you 
might  carry  an  iButton  with  you  (attached  to  a 

ring,  or  a  key,  or  a  badge)  and  the  Tini  board  reads  it  and  personalizes  its 
environment  to  you. 

Real-Time  Java 

Because  the  extensive  Java  infrastructure  that  has  been  built  (in  the  form  of 
Ptolemy  II),  we  have  been  studying  the  various  real-time  Java  projects  with  an 
eye  towards  being  able  to  provide  native  Java  support  for  the  OCP,  including  its 
quality  of  service  features.  Particular  realizations  we  have  studied  include 
Esmertec,  FIXME. 

CORBA  Sensors 

Brian  Vogel  created  a  demonstration  that  uses  CORBA  to  obtain  and  plot  data 
from  an  ultrasonic  detector  connected  to  a  Windows  NT  machine  running  a  web 
server.  He  uses  the  audio  hardware  of  the  PC  as  an  analog  interface  to  the 
sensor.  This  was  our  first  networked  sensor  application,  and  it  illustrates  how 
smart  sensors  might  communicate  with  controllers  via  the  OCP. 

Controller  Design  and  Analysis 

We  did  extensive  research  on  hybrid  system  theory  which  enable  multi-agent 
multi-modal  control.  The  objective  is  to  use  hybrid  system  model  for  the  design, 
analysis  and  synthesis  of  multi-agent  multi-modal  systems  that  deliver  high  levels 
of  mission  reliability  in  dynamic  and  rapidly  evolving  environments. 

Multi-Modal  Control  Derivation 

John  Koo,  Shankar  Sastry,  and  George  Pappas  of  University  of  Pennsylvania 
worked  on  mode  switching  synthesis  for  specifications  which  are  based  on 
mission  completion  properties.  In  particular,  we  were  interested  in  reachability 
specifications.  The  objective  was  to  develop  a  general  framework  for  the  study  of 
control  mode  switching  and  algorithms  for  the  derivation  of  sequences  of  control 
modes  which  will  satisfy  reachability  tasks.  We  implemented  the  algorithms 
based  on  existing  efficient  methods  for  reachability  computation  and  apply  the 
algorithms  to  certain  classes  of  dynamical  systems. 

Component-based  Design  Technique  for  Multi-Modal  Control 

Based  on  previous  works  by  Jie,  Xiaojun,  John  Koo  and  Bruno  Sinopoli  on  the 
implementation  of  a  multi-modal  system  with  fixed  mode  sequences  as  a 
hierarchical  hybrid  system  in  Ptolemy  II,  we  investigated  the  use  of  the  proposed 
component-based  design  technique  to  implement  a  scaled  down  version  of  UAV 
flight  management  system  which  facilitates  on-line  control  mode  sequence 
generation.  The  proposed  mode  switching  algorithm  for  reachability 
specifications  will  be  used  to  demonstrate  the  versatility  of  the  proposed  software 
design  technique  for  the  design  of  embedded  software  which  enables  multi¬ 
modal  control. 
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Computations  for  Optimal  Hybrid  Control 

Rene  Vidal,  Shawn  Schaffert,  John  Lygeros  and  Shankar  Sastry  have  developed 
algorithms  for  computing  the  maximal  controlled  invariant  set  and  least  restrictive 
controller  for  discrete  time  systems.  We  show  that  the  algorithm  can  be  encoded 
using  quantifier  elimination,  which  leads  to  a  semi-deciablity  result  for  definable 
system.  Rene  Vidal  and  Omid  Shakernia  have  implemented  an  algorithm  for  a 
least  restrictive  controller  synthesis  based  on  robust  semidefinite  programming. 
Omid  Shakernia  has  shown  theoretical  result  on  decidable  controller  synthesis 
for  classes  of  linear  systems. 

Modeling  of  Hybrid  Systems 

The  objective  of  the  work  was  to  have  a  deeper  and  border  understanding  of 
dynamical  behaviors  of  hybrid  systems  in  order  to  benefit  the  development  of 
new  algorithms  for  analysis,  synthesis,  verification  and  simulation  of  hybrid 
systems.  Jun  Zhang,  Karl  Johansson,  John  Lygeros  and  Shankar  Sastry  have 
extended  several  important  results  of  classical  dynamical  systems  to  hybrid 
dynamical  systems,  especially  those  with  Zeno  executions.  Slobodan  Simic,  Karl 
Johansson,  John  Lygeros  and  Shankar  Sastry  have  introduced  a  new  framework 
for  a  global  geometric  study  of  hybrid  systems.  Its  usefulness  is  demonstrated  in 
analysis  of  the  Zeno  phenomenon  and  stability  of  hybrid  equilibria.  Ekaterina 
Lemch  has  studied  the  controllability  of  hybrid  system  in  this  geometric 
framework. 

Probabilistic  Hybrid  Systems 

Fault  detection  and  handling  need  to  be  designed  so  as  to  guarantee  a 
hierarchical  decrease  in  functionality  from  the  least  safety-critical  to  the  most. 
Off-line,  the  fault  handling  routines  need  to  be  analyzed  and  verified  in  a 
probabilistic  framework.  Jianghai  Hu,  John  Lygeros  and  Shankar  Sastry  have 
extended  the  deterministic  framework  of  hybrid  system  to  the  nondeterministic 
one.  This  provides  an  analytical  framework  for  the 
verification  of  fault  detection  and  handling. 

UAV  Control  Software  Development 

David  Shim,  Xiaojun  and  John  Koo  has  been  working  on  formal  modeling  of 
existing  embedded  software  into  Unified  Modeling  Language.  The  objective  is  to 
capture  the  organization  of  software  components,  scheduling  scheme  provided 
by  operating  systems  and  temporal  properties  and  constraints  of  different 
hardware  and  software  components. 
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Chapter  Three:  Accomplishments  in  2001 


Our  effort  integrated  three  tasks:  formal  modeling  and  verification; 
embedded  software  design;  and  controller  design  and  analysis.  On  the  formal 
side,  we  introduced  a  hierarchical  model  for  the  design  and 
analysis  of  embedded  control  systems  called  Masaccio.  This  will  be  the  basis  for 
the  next  generation  of  the  hybrid  model  checker  Hytech.  We  also  focused 
heavily  on  the  study  of  games  for  synthesizing  controllers.  Our  original  results 
concern  the  solution  of  probabilistic  games  (for  modeling  synchronous  control 
and  uncertain  environments)  and  abstraction  mechanisms  for  games.  At  the  PI 
meeting  in  Albuquerque,  we  demonstrated  hierarchical  heterogeneous  modeling 
and  design  using  Ptolemy  II,  with  integrated  smart  sensors  and  motor-drive 
actuators  to  control  small  robots.  We  also  described  a  preliminary 
implementation  of  a  time-triggered  language,  and  progress  on  its  underlying 
formalism.  On  the  control  side,  we  showed  that  the  controller  synthesis  problem 
is  semi-decidable  for  special  classes  of  hybrid  systems  where  the  continuous 
vector  fields  are  linear,  and  we  obtained  original  results  in  the  optimal  control  of 
switched  linear  systems. 

Formal  Modeling  and  Analysis 

We  defined  a  hierarchical  model  for  the  design  and  analysis  of  embedded 
control  systems  called  Masaccio.  The  next  generation  of  Hytech  is  planned  to 
operate  on  Masaccio  models.  We  also  focused  heavily  on  the  study  of  games. 
Games  are  the  natural  setting  for  studying  the  synthesis  of  controller:  a  controller 
is  a  winning  strategy  of  a  game  system  vs.  control.  Our  original  results  concern 
the  solution  of 

probabilistic  games  (for  modeling  synchronous  control  and  uncertain 
environments)  and  abstraction  mechanisms  for  games.  To  our  knowledge,  our 
work  on  abstraction  of  games  is  the  first  attempt  at  an  abstract  interpretation 
theory  for  controller  synthesis. 

Masaccio:  a  formal  model  for  embedded  components 

We  defined  Masaccio,  a  formal  model  for  hybrid  dynamical  systems  which  are 
built  from  atomic  discrete  components  (difference  equations)  and  atomic 
continuous  components  (differential  equations)  by  parallel  and  serial 
composition,  arbitrarily  nested.  Each  system  component  consists  of  an  interface, 
which  determines  the  possible  ways  of  using  the  component,  and  a  set  of 
executions,  which  define  the  possible  behaviors  of  the  component  in  real  time. 
The  purpose  of  Masaccio  is  to  enable  hierarchical  formal  modeling  and 
verification  of  hybrid  systems. 

Concurrent  omega-regular  games 

We  consider  two-player  games  which  are  played  on  a  finite  state  space  for  an 
infinite  number  of  rounds.  The  games  are  concurrent,  that  is,  in  each  round,  the 
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two  players  choose  their  moves  independently  and  simultaneously;  the  current 
state  and  the  two  moves  determine  a  successor  state.  We  consider  omega- 
regular  winning  conditions  on  the  resulting  infinite  state  sequence.  To  model  the 
independent  choice  of  moves,  both  players  are  allowed  to  use  randomization  for 
selecting  their  moves.  This  gives  rise  to  the  following  qualitative  modes  of 
winning,  which  can  be  studied  without  numerical  considerations  concerning 
probabilities:  sure-win  (player  1  can  ensure  winning  with  certainty),  almost-sure- 
win  (player  1  can  ensure  winning  with  probability  1),  limit-win  (player  1  can 
ensure  winning  with  probability  arbitrarily  close  to  1),  bounded-win  (player  1  can 
ensure  winning  with  probability  bounded  away  from  0),  positive-win  (player  1  can 
ensure  winning  with  positive  probability),  and  exist-win  (player  1  can  ensure  that 
at  least  one  possible  outcome  of  the  game  satisfies  the  winning  condition). 

We  provide  algorithms  for  computing  the  sets  of  winning  states  for  each  of  these 
winning  modes.  In  particular,  we  solve  concurrent  Rabin-chain  games  in  nA0(m) 
time,  where  n  is  the  size  of  the  game  structure  and  m  is  the  number  of  pairs  in 
the  Rabin-chain  condition.  While  this  complexity  is  in  line  with  traditional  turn- 
based  games,  where  in  each  state  only  one  of  the  two  players  has  a  choice  of 
moves,  our  algorithms  are  considerably  more  involved  than  those  for  turn-based 
games.  This  is  because  concurrent  games  violate  two  of  the  most  fundamental 
properties  of  turn-based  games.  First,  concurrent  games  are  not  determined,  but 
rather  exhibit  a  more  general  duality  property  which  involves  multiple  modes  of 
winning.  Second,  winning  strategies  for  concurrent  games  may  require  infinite 
memory. 

Abstract  interpretation  of  game  properties 

We  apply  the  theory  of  abstract  interpretation  to  the  verification  of  game 
properties  for  reactive  systems.  Unlike  properties  expressed  in  standard 
temporal  logics,  game  properties  can  distinguish  adversarial  from  collaborative 
relationships  between  the  processes  of  a  concurrent  program,  or  the  components 
of  a  parallel  system.  We  consider  two-player  concurrent  games  — say, 
component  vs.  environment —  and  specify  properties  of  such  games  — say,  the 
component  has  a  winning  strategy  to  obtain  a  resource,  no  matter  how  the 
environment  behaves —  in  the  alternating-time  mu-calculus  (AMU).  A  sound 
abstraction  of  such  a  game  must  at  the  same  time  restrict  the  behaviors  of  the 
component  and  increase  the  behaviors  of  the  environment:  if  a  less  powerful 
component  can  win  against  a  more  powerful  environment,  then  surely  the  original 
component  can  win  against  the  original  environment. 

We  formalize  the  concrete  semantics  of  a  concurrent  game  in  terms  of 
controllable  and  uncontrollable  predecessor  predicates,  which  suffice  for  model 
checking  all  AMU  properties  by  applying  boolean  operations  and  iteration.  We 
then  define  the  abstract  semantics  of  a  concurrent  game  in  terms  of  abstractions 
for  the  controllable  and  uncontrollable  predecessor  predicates.  This  allows  us  to 
give  general  characterizations  for  the  soundness  and  completeness  of  abstract 
games  with  respect  to  AMU  properties.  We  also  present  a  simple  programming 
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language  for  multi-process  programs,  and  show  how  approximations  of  the 
maximal  abstraction  (w.r.t.  AMU  properties)  can  be  obtained  from  the  program 
text.  We  apply  the  theory  to  two  practical  verification  examples,  a 
communication  protocol  developed  at  the  Berkeley  Wireless  Research  Center, 
and  a  protocol  converter.  In  the  wireless  protocol,  both  the  use  of  a  game 
property  for  specification  and  the  use  of  abstraction  for  automatic  verification 
were  instrumental  to  uncover  a  subtle  bug. 

Embedded  Software  Design 

The  objective  of  this  part  of  the  project  is  to  adapt  sophisticated 
software  techniques  to  the  problem  domain  of  real-time  control  systems. 

The  problem  domain  mandates  a  priority  on  real-time  guarantees  and  verifiable 
designs.  Our  approach  is  based  on  formalizing  the  models  of  computation  that 
govern  the  interaction  between  components  (both  hardware  and  software 
components),  and  building  software  that  realizes  these  models  of  computation. 
Control  systems  integrate  a  variety  of  hardware,  including  networked  sensors 
and  actuators,  with  variety  of  service  demands,  ranging  from  hard-real-time 
signal  processing  to  interaction  with  human  operators.  Hence,  our  approach 
emphasizes  the  composition  of  heterogeneous  models  of  computation. 

For  example,  publish-and-subscribe  interactions  between  components 
(with  priorities)  are  suitable  for  managing  irregular  events  and 
alarms,  but  not  so  well  suited  for  regular,  high-sample-rate  signal 
processing  nor  for  continuous  dynamics.  Time-triggered  formalisms  are 
better  suited  for  hard-real-time  processing,  but  not  as  well  suited  for 
irregular  interactions.  Our  approach,  therefore,  is  to  study  the 
composition  of  these  models  and  to  build  software  that  enables 
systematic  composition.  Another  example  of  such  heterogeneous 
compositions  uses  state  machine  models  to  govern  modes  of  operation  of 
systems  whose  behavior  is  better  represented  in  some  other  way,  such  as 
by  continuous-time  models,  to  get  hybrid  systems. 

We  are  applying  modern  software  concepts,  including  sophisticate  approaches  to 
concurrency  management,  discovery,  middleware,  type  systems,  and  higher- 
order  functions  to  this  problem  domain.  We  are  using  a  pre-existing  Java-based 
software  framework  called  Ptolemy  II  as  an  experimental  platform,  and  we  are 
working  with  the  Berkeley  Aerobots  project  to  put  our  software  concepts  on 
autonomous  model  helicopters. 

Fresco 

Ben  Horowitz  and  Christoph  Meyer  have  finished  a  draft  implementation 
of  a  skeleton  of  Giotto  in  VxWorks.  Giotto  is  our  evolving  time-triggered 
implementation  language.  We  intend  the  implementation  to 
be  portable  between  RTOSs  (e.g.,  Real-time  Linux,  QNX,  etc.)  In 
particular,  we  have  limited  our  use  to  POSIX  features.  We  found  the 
logic  analyzer  of  VxWorks  very  helpful,  both  for  debugging  and  for 
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understanding  the  behavior  of  the  RTOS. 

There  is  a  screen  shot  available  at 

http://www-cad.eecs.berkeley.edu/~fresco/teos/teos3-log-2.gif, 

which  serves  as  a  good  example.  In  it,  one  can  see  four  processes  of 
interest: 

-  com,  which  is  responsible  for  intertask  communication 

-  taskO,  which  executes  four  times  per  round 

-  taskl ,  which  executes  two  times  per  round 

-  task2,  which  executes  once  per  round 

There  is  also  an  idle  task,  which  is  not  of  interest.  Here's  what's 
going  on: 

-  at  time  12,  the  communication  task  begins  to  run.  It  makes 
taskO,  taskl ,  and  task2  ready  to  run.  The  heavy  green  horizontal 
line  signifies  that  a  task  is  running,  and  the  sawtooth  horizontal 
line  signifies  that  a  task  is  ready. 

-  at  time  22,  taskO  begins  to  run,  and  finishes  shortly  thereafter. 

-  at  time  23,  taskl  begins  to  run.  It  finishes  at  time  34. 

-  at  time  34,  task2  begins  to  run.  It  gets  preempted  by  com  at  time 
43. 

-  at  time  50,  taskl  begins  to  run,  and  finishes  shortly  thereafter. 

-  at  time  52,  task2  resumes,  and  finishes  at  time  61 . 


Note  that  task2  continues  to  run  across  multiple  invocations  of  taskO. 

This  is  possible  because  of  the  preemptive  scheduler  of  VxWorks. 

Albuquerque  Demo 

At  the  PI  meeting  in  Albuquerque,  we  demonstrated  hierarchical  heterogeneous 
modeling  and  design  using  Ptolemy  II  to  integrate  smart 
sensors  and  motor-drive  actuators  to  control  small  robots.  The 
implementation  used  a  publish-and-subscribe  fabric  (JavaSpaces),  a 
service  discovery  fabric  (Jini),  a  discrete-event  simulation  engine 
(the  Ptolemy  II  DE  domain),  and  a  dataflow  computation  engine  (the 
Ptolemy  II  SDF  domain).  It  showed  models  that  were  modified  during 
execution,  with  thread-safe  updates  of  static  analysis  information.  The 
discovery  mechanism  was  used  to  dynamically  discover  the  publish  and 
subscribe  fabric.  We  believe  that  the  same  infrastructure  can  be  used 
to  discover  other  publish-and-subscribe  fabrics,  such  as  the  OCP. 

The  small  robots  were  Lego  Mindstorms,  and  the  sensor  was  a  tilt  sensor 
from  Telemonitor  connected  via  an  IEEE  1451 .2  bus  to  an  Agilent  NCAP 
(network  capable  application  processor),  which  hosted  a  web  server.  The 
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robots  and  the  sensors  were  abstracted  as  Ptolemy  II  actors.  In  the 
case  of  the  robots,  the  actor  functioned  as  a  proxy,  internally 
implemented  by  sending  commands  to  the  robot  over  the  serial  port  and 
an  infrared  link.  To  improve  reliability  and  directionality  of  the 
infrared  link,  we  used  a  port  replicator  (designed  and  built  by  Win 
Williams)  to  drive  two  distinct  infrared  emitters.  The  Telemonitor 
sensors  were  abstracted  as  actors  that  performed  HTTP  queries  of  the 
NCAP  to  obtain  sensor  information  and  to  configure  the  sensors. 

A  number  of  issues  were  raised  by  this  demo,  particularly  with  regard 
to  the  semantics  of  the  publish-and-subscribe  infrastructure.  There  are 
a  number  of  options  in  configuring  this  infrastructure.  For  instance: 

-  Should  events  have  time  stamps? 

-  Should  these  time  stamps  have  global  significance? 

-  Should  delivery  of  events  be  reliable  (with  retry  if  necessary)? 

-  Should  events  always  be  delivered  in  the  order  they  are  produced? 

-  How  can  we  coordinate  events  from  multiple  sources? 

-  Should  synchronous  event  delivery  be  supported? 

-  How  should  events  from  multiple  publishers  be  merged? 

-  How  should  dynamic  redirection  and  resourcing  of  events  be  supported? 

-  Should  the  fabric  provide  persistent  state? 

-  Should  the  fabric  provide  history  services? 

In  view  of  these  questions,  the  next  step  is  to  define  a  precise 
semantics  for  a  publish-and-subscribe  fabric  for  software-enabled 
control  systems. 

The  demo  builders  were  Chamberlain  Fong,  Christopher  Hylands,  Jie  Liu, 
Xiaojun  Liu,  Steve  Neuendorffer,  Sonia  Sachs,  and  Win  Williams. 

Discovery 

At  the  Albuquerque  PI  meeting,  we  also  proposed  the  use  of  discovery 
mechanisms  (such  as  Jini)  to  construct  a  "Meta-OCP,"  where  various 
configurations  of  interconnection  services  could  be  delivered  in  a 
modular  way.  For  example,  a  component  may  request  a  reliable  stream-based 
delivery  mechanism  to  get  sampled  data  from  point  A  to  point  B.  The  meta-OCP 
would  respond  with  byte-code  that  implements  a  suitable  interface,  using  for 
example  TCP/IP  and  sockets,  bypassing  any  central 
infrastructure.  This  might  be  used  to  transport  regularly  sampled  data, 
such  as  audio  data.  In  another  example,  a  component  requests  a  shared  data 
repository  visible  to  a  number  of  components.  The  delivered  code  might  interact 
with  a  Linda-style  tuple  space,  such  as  that  in  the  OCP.  This  might  be  used,  for 
example,  to  read  the  current  temperature  from  a  sensor.  In  a  third  example,  a 
component  requests  an  interface  to  send 

time-stamped  data  that  must  be  delivered  and  dealt  with  within  some 
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specified  deadline.  The  delivered  code  might  interact  with  TAO  via  the 
OCP.  This  might  be  used,  for  example,  to  deliver  motion  control  data. 

Controller  Design  and  Analysis 

We  have  done  extensive  research  on  hybrid  system  theory  which  enable 
multi-agent  multi-modal  control.  The  objective  is  to  use  hybrid  system  model  for 
the  design,  analysis  and  synthesis  of  multi-agent  multi-modal  systems  that 
deliver  high  levels  of  mission  reliability  in  dynamic  and  rapidly  evolving 
environments.  For  controller  synthesis,  we  have  worked  on  multi-modal  control 
derivation  and  computations  for  optimal  hybrid  control.  The  objective  of  multi¬ 
modal  control  research  is  to  develop  a  general  framework  for  the  study  of  control 
mode  switching  and  algorithms  for  the  derivation  of  sequences  of  control  modes 
which  will  satisfy  reachability  tasks.  Synthesis  of  least  restrictive  controllers 
based  on  optimal  control  theory  for  classes  of  systems  in  which  the  reachability 
problem  are  deciable  are  undertaking.  Modeling  of  hybrid  systems  in  both 
deterministic  and  nondeterministic  settings  are  currently  being  investigated.  The 
objective  of  the  studies  is  to  have  a  deeper  and  border  understanding  of 
dynamical  behaviors  of  hybrid  systems  in  order  to  benefit  the  development  of 
new  algorithms  for  analysis,  synthesis,  verification  and  simulation.  Component- 
based  design  technique  will  be  deployed  for  the  design  of  embedded 
software  which  enables  multi-modal  control.  In  parallel,  formal  modeling 
language,  unified  modeling  language,  is  used  for  modeling  of  existing  embedded 
software  running  on  helicopter-based  unmanned  aerial  vehicle  developed  at 
Berkeley  to  motivate  the  study  in  applying  component-based  design  techniques 
for  real-time  embedded  system. 

Semi-decidable  Controller  Synthesis  for  Classes  of  Linear  Hybrid  Systems 

A  problem  of  great  interest  in  the  control  of  hybrid  systems  is  the  design  of  least 
restrictive  controllers  for  reachability  specifications.  Controller  design  typically 
uses  game  theoretic  methods  to  compute  the  region  of  the  state  space  for  which 
there  exists  a  control  such  that  for  all  disturbances,  an  unsafe  set  is  not  reached. 
In  general,  the  computation  of  the  controllers  requires  the  steady  state  solution  of 
a  Hamilton-Jacobi  partial  differential  equation  which  is  very  difficult  to  compute,  if 
it  exists.  We  show  that  for  special  classes  of  hybrid  systems  where  the  continous 
vector  fields  are  linear,  the  controller  synthesis  problem  is  semi-decidable:  There 
exists  a  computational  algorithm  which,  if  it  terminates  in  a  finite  number  of  steps, 
will  exactly  compute  the  least  restrictive  controller.  This  result  is  achieved  by  a 
very  interesting  interaction  of  results  from  mathematical  logic  and  optimal  control 
and  presents  us  with  the  first  semi-decidable  controller  synthesis  result  for  such 
rich  classes  of  hybrid  systems. 

Toward  Optimal  Control  of  Switched  Linear  Systems 

The  problem  of  driving  the  states  of  a  switched  linear  control  system  between 
boundary  states  is  investigated.  We  propose  tight  lower  bounds  for  the  minimum 
energy  control  problem.  We  furthermore  show  that  the  fact  that  the  dynamics  is 
changing  discontinuously  on  the  switching  surface  gives  rise  to  phenomena  that 
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can  be  treated  as  a  decidability  problem  for  hybrid  control  systems,  using  ideas 
from  mathematical  logic. 
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Chapter  Four:  Accomplishments  in  2002 


Giotto 

We  continued  development  on  the  time-triggered  Giotto  model  of  computation. 
Specifically,  we  are  investigating  the  schedulability  of  Giotto  on  distributed 
platforms  (Ben  Horowitz)  and  we  have  implemented  a  Giotto  compiler  on  a  UAV 
rotorcraft  from  ETH  Zurich  (Christoph  Kirsch,  and  Marco  Sanvido  drom  ETH).  All 
ground  tests  of  Giotto  on  the  model  helicopter  have  been  successfully  completed 
and  we  are  currently  awaiting  some  sort  of  certification  for  flying.  Giotto 
implements  the  glue  (scheduling)  code  for  the  entire  flight  control  systems,  which 
manages  the  execution  of  native  software  tasks  written  in  Oberon. 

The  first  Giotto  paper  was  published  in  the  ACM  Workshop  for  Languages, 
Compilers,  and  Tools  for  Embedded  Systems:  Giotto  is  a  principled,  tool- 
supported  design  methodology  for  implementing  embedded  control  systems  on 
platforms  of  possibly  distributed  sensors,  actuators,  CPUs,  and  networks.  Giotto 
is  based  on  the  principle  that  time-triggered  task  invocations  plus  time-triggered 
mode  switches  can  form  the  abstract  essence  of  programming  real-time  control 
systems.  Giotto  consists  of  a  programming  language  with  a  formal  semantics, 
and  a  retargetable  compiler  and  runtime  library.  Giotto  supports  the  automation 
of  control  system  design  by  strictly  separating  platform-independent  functionality 
and  timing  concerns  from  platform-dependent  scheduling  and  communication 
issues.  The  time-triggered  predictability  of  Giotto  makes  it  particularly  suitable  for 
safety-critical  applications  with  hard  real-time  constraints. 

Dynamic  Programming  Algorithms  for  Controller  Synthesis 

Symbolic  model  checking  algorithms  for  system  verification  can  be  viewed  as 
dynamic  programs.  We  showed  that  the  same  dynamic  programming  schemes 
can  also  be  used  for  the  automatic  synthesis  of  optimal  control  strategies.  This 
work  lies  the  foundation  for  using  a  model-checking  tool  such  as  HyTech  for 
controller  synthesis.  We  published  these  results  at  the  2001  IEEE  Symposium 
on  Logic  in  Computer  Science. 

Dynamic  programs,  or  fixpoint  iteration  schemes,  are  useful  for  solving  many 
problems  on  state  spaces,  including  model  checking  on  Kripke  structures 
("verification"),  computing  shortest  paths  on  weighted  graphs  ("optimization"), 
computing  the  value  of  games  played  on  game  graphs  ("control").  For  Kripke 
structures,  a  rich  fixpoint  theory  is  available  in  the  form  of  the  mu-calculus.  Yet 
few  connections  have  been  made  between  different  interpretations  of  fixpoint 
algorithms.  We  study  the  question  of  when  a  particular  fixpoint  iteration  scheme  f 
for  verifying  an  omega-regular  property  L  on  a  Kripke  structure  can  be  used  also 
for  solving  a  two-player  game  on  a  game  graph  with  winning  objective  L.  We 
provide  a  sufficient  and  necessary  criterion  for  the  answer  to  be  affirmative  in  the 
form  of  an  extremal-model  theorem  for  games:  under  a  game  interpretation,  the 
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dynamic  program  f  solves  the  game  with  objective  L  if  and  only  if  both  (1 )  under 
an  existential  interpretation  on  Kripke  structures,  f  is  equivalent  to  EL,  and 
(2)  under  a  universal  interpretation  on  Kripke  structures,  f  is  equivalent  to  AL.  In 
other  words,  f  is  correct  on  all  two-player  game  graphs  iff  it  is  correct  on  all 
extremal  game  graphs,  where  one  or  the  other  player  has  no  choice  of  moves. 
The  theorem  generalizes  to  quantitative  interpretations,  where  it  connects  two- 
player  games  with  costs  to  weighted  graphs. 

While  the  standard  translations  from  omega-regular  properties  to  the 
mu-calculus  violate  (1)  or  (2),  we  give  a  translation  that  satisfies  both  conditions. 
Our  construction,  therefore,  yields  fixpoint  iteration  schemes  that  can  be 
uniformly  applied  on  Kripke  structures,  weighted  graphs,  game  graphs,  and 
game  graphs  with  costs,  in  order  to  meet  or  optimize  a  given  omega-regular 
objective. 

E-Machine 

We  have  initiated  an  effort  to  define  a  "virtual  machine"  layer  that 
abstracts  the  real-time  and  task  interaction  properties  of  an  embedded 
system.  The  intent  of  the  virtual  machine  is  to  serve  as  a  target  for 
a  problem-oriented  "controls  API,"  and  to  be  implementable  on  a  variety 
of  hardware/software  architectures,  including  the  current  Boeing  OCP. 

The  virtual  machine  layer  concept  is  our  proposal  to  reconcile  the 
needs  of  distributed  supervisory  control,  which  matches  a  publish  and 
subscribe  style  of  interaction  using  real-time  CORBA,  with  lower-level 
hard-real-time  control,  which  matches  time-driven  models  such  as  FRP 
(from  Yale)  and  Giotto  (from  Berkeley).  At  the  same  time,  the  virtual 
machine  is  attempting  to  address  issues  of  concurrency  management, 
particularly  with  respect  to  precise  reactions  and  precise  mode 
switching,  in  a  way  that  is  independent  of  the  level  of  control  being 
applied.  Thus,  the  same  conceptual  framework  can  be  shared. 

A  first  attempt  at  this  virtual  machine  definition  has  been  carried  out 
by  Christoph  Kirsch  and  Tom  Henzinger,  and  is  described  at: 

h  ttp :  //www .  eecs .  be  rke  I  ey .  ed  u/~f  resco/g  iotto/refs 

Their  virtual  machine  is  called  the  E-machine,  for  "embedded  machine." 

A  program  in  this  virtual  machine  consists  of  sets  of  tasks  that  take 
time  and  cannot  be  logically  interrupted.  There  are  no  synchronization 
points  between  these  tasks,  thus  avoiding  a  key  source  of  problems  with 
deadlock  and  unexpected  nondeterminacy  in  concurrent  programs.  Instead, 
interactions  between  tasks  are  through  ports.  Input  ports  provide 
values  that  do  not  change  during  execution  of  the  task.  Output  ports 
are  updated  only  at  the  completion  of  the  task.  This  is  key  to  the 
operation  of  the  virtual  machine,  and  provides  the  principle  mechanism 
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for  achieving  predictable  behavior. 

Tasks  take  time  to  execute,  and  execute  concurrently  with  other  tasks. 

The  virtual  machine  does  not  define  the  underlying  task  scheduling 
model.  That  scheduling  model  would  typically  be  provided  by  an 
underlying  real-time  OS.  A  dispatcher  enables  tasks  for  execution,  and 
runs  at  the  highest  priority.  The  only  assumption  made  about  the 
underlying  task  scheduling  model  is  that  if  the  dispatcher  has  work  to 
do,  then  it  will  execute,  preempting  all  other  tasks. 

In  addition  to  tasks  that  take  time,  the  virtual  machine  defines 
"drivers,"  which  are  tasks  that  conceptually  execute  instantaneously. 

These  execute  at  the  priority  of  the  dispatcher. 

Time  information  is  provided  by  the  environment  through  ports,  and 
these  ports  can  trigger  the  enabling  of  tasks.  Ports  can  also  be  used 
to  signal  the  completion  of  a  task,  and  activation  of  tasks  can  depend 
on  port  values. 

An  E  machine  is  an  interpreter  for  E  code.  A  compiler  could  generate 
the  E  machine  interpreter,  including  the  task  manager  (RTOS 
generation). 

Jie  Liu  has  developed  a  proposal  for  a  distributed  E  machine  interpreter  built 
using  the  Boeing  OCP.  This  is  an  evolving  proposal,  and  discussions  with 
Boeing  are  under  way. 

Virtual  Machine  Working  Group 

On  June  13,  we  held  an  all  day  meeting  at  Berkeley  to  get  inputs 
from  the  technology  developers  in  SEC  on  the  virtual  machine  concept. 
Attending  were:  John  Peterson,  Yale,  Richard  Kieburtz,  OGI,  Jie  Liu,  Berkeley, 
Walid  Taha,  Yale,  Johan  Nordlander,  OGI,  Tom  Henzinger,  Berkeley,  Edward  A. 
Lee,  Berkeley,  Ben  Horowitz,  Berkeley,  Magnus  Carlsson,  OGI,  Joern  Janneck, 
Berkeley,  and  Xiaojun  Liu,  Berkeley 

Jie  Liu  gave  an  overview  of  the  OCP,  and  we  discussed  a  virtual  machine 
interpretation  of  the  OCP.  Johan  Nordlander  described  the  object  model 
underlying  O'Haskell  (now  called  Timber),  which  we  identified  as  a 
programming  model  rather  than  a  virtual  machine.  Magnus  Carlsson 
described  an  alternative  to  the  OCP  event  channel  for  task  interaction. 

Tom  Henzinger  described  his  strawman  virtual-machine  design,  and  argued 
that  it  offers  to  the  virtual  machine  concept  several  key  ideas: 

-  atomic  execution  ("drivers") 

-  ability  to  have  a  distributed  time  model. 

-  Determinism  in  two  dimensions: 
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+  Time  it  takes  to  do  something  (delay  committing  results) 

+  Values  of  inputs  and  outputs  (double  buffering) 

Jie  Liu  described  our  hierarchical  preemptive  multitasking  (HPM)  domain 
in  Ptolemy  II,  and  we  identified  a  realization  of  this  domain  on  the 
strawman  virtual  machine.  This  domain  offers  a  solution  to  the  precise 
mode  change  problem  in  the  OCP,  and  shares  with  the  E-machine  the  split 
phase  read/commit  of  inputs  and  outputs. 

John  Peterson  described  FRP,  which  he  identified  as  low-level  semantics 
work.  FRP  adds  time  flow  to  any  existing  language.  It  is  possible  that 
FRP  could  be  used  to  represent  the  semantics  of  the  virtual  machine. 

Walid  Taha  described  real-time  FRP,  an  abstract  subset  on  FRP  with 
guaranteed  bounds  on  execution  time  and  space. 

We  identified  the  following  action  items: 

-  Sell  the  split  phase  execution  concept. 

+  Solves  the  precise  mode  change  problem. 

+  Introduces  a  measure  of  determinism. 

-  Define  the  embedded  machine 
+  With  a  better  syntax 

+  Top-level  control  loop 
+  An  informal  operational  semantics 
+  Define  the  task  model 
+  Type  system  (not  in  the  critical  path) 

+  Distribution  model  (&  connection  to  event  channel  model) 

+  Prototype  implementation 

-  Show  how  InputDataAvailableQ  can  be  converted  to  a  split  phase  execution. 
+  Backward  compatible  version  might  assume  this  is  a  driver. 

-  Show  have  various  programming  models  map  onto  the  EM. 

+  Concurrent  real-time  FRP 

+  HPM  (hierarchical  preemptive  multitasking) 

+  Giotto 

+  Priority-driven  publish/subscribe 
+  Timber 

+  Rate  monotonic  scheduling 

Blending  Controllers 

Linda  Wills  and  Bonnie  Heck  of  Georgia  Tech  have  been  working  on 
"blending  controllers,"  where  during  a  mode  change,  the  control  laws  of 
two  controllers  are  blended  in  a  controlled  way  during  the  transition 
between  the  two  controllers.  These  are  modal  (hybrid)  controllers  where 
instead  of  abruptly  switching  from  one  control  law  to  another,  an 
intermediate  "blending"  mode  is  use  to  gradually  turn  over  control  from 
one  to  the  other. 
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We  began  working  with  them  on  exploring  ways  of  realizing  these 
controllers,  first  via  conference  calls,  and  then  when  Linda  Wills  and 
Murat  Guller  visited  us  on  6/5/01  and  6/6/01 .  They  posed  the  question 
of  how  to  identify  a  suitable  programmer's  model  for  such  modal 
controllers.  We  explored  various  alternatives,  including  mechanisms 
that  could  be  built  immediately  in  Ptolemy  II,  but  found  none  of  them 
suitable.  We  then  discovered  that  a  small  change  to  the  FSM  (finite 
state  machine)  domain  in  Ptolemy  II  would  enable  a  very  clean 
programmer's  model.  In  this  model,  a  three  state  FSM  is  defined  where 
in  the  first  state,  controller  A  is  active  and  in  the  third  state, 
controller  B  is  active.  In  the  second  state,  both  controllers  are 
active  and  their  outputs  are  blended  using  an  expression.  The  small 
change  required  in  the  FSM  domain  is  that  states  must  be  able  to  have 
more  than  one  active  refinement.  This  turned  out  to  be  relatively 
easily  to  implement.  Xiaojun  Liu  and  Jie  Liu  completed  an 
implementation  in  about  one  hour,  and  we  constructed  a  simple 
demonstration  system  in  about  another  half  hour. 

Generalized  Modal  Models 

An  outcome  of  the  meeting  with  Georgia  Tech  (see  above)  was  the 

realization  that  the  Ptolemy  II  notion  of  a  modal  controller  can  be 

usefully  generalized  to  one  where  a  mode  can  have  more  than  one  (concurrent) 

refinement.  The  finite-state  machine  governing  the  mode 

changes  can  then  be  viewed  as  providing  a  mask,  making  some  subset  of 

the  actors  of  a  model  visible  to  the  director.  This  appears  to  be  a 

significant  increase  in  expressiveness  over  our  prior  *-charts 

formalism,  which  is  a  generalization  of  Statecharts  and  hybrid  systems. 

Some  possible  applications  of  this  formalism: 

1)  It  could  be  used  for  managing  redundant  systems  and  doing 
fault  detection  and  isolation. 

2)  It  could  be  that  heterochronous  dataflow  is  a  special  case. 

3)  It  could  be  viewed  as  a  structured  mutations  approach. 

Xiaojun  Liu  is  exploring  this  concept  further. 

HPM  (formerly  RTOS)  Domain 

Jie  Liu  has  made  major  progress  on  his  model  of  computation  supporting 
precise  reactions  in  the  context  of  preemptive  multitasking.  In  the 
previous  reporting  period,  we  reported  a  Ptolemy  II  domain  called  the 
RTOS  domain  that  modeled  RTOS-style  preemptive  multitasking.  Jie  has 
elaborated  the  semantics  of  this  domain  in  very  interesting  ways,  and 
has  come  up  with  a  new  model  of  computation  that  is  significantly 
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different  from  the  previous  version.  We  are  calling  this  domain  the  HPM 
domain,  for  hierarchical  preemptive  multitasking. 

The  key  idea  in  HPM  is  that  of  split-phase  firing,  inspired  by  the 
Giotto  model.  In  this  manifestation,  actors  read  and  locally  cache 
input  data  when  their  prefire()  method  is  invoked,  and  (optionally) 
begin  computation  in  a  background  process  that  executes  concurrently 
with  other  processes  with  some  specified  priority.  Not  until  the  fire() 
method  is  invoked,  however,  do  these  actors  write  output  data.  In  Jie's 
model,  the  actors  declare  their  execution  time,  and  the  fire()  method 
is  invoked  after  they  have  had  available  CPU  execution  time  equal  to 
this  declared  value.  A  dispatcher  process  keeps  track  of  invocations  of 
actors  with  higher  priority  so  that  the  process  is  assured  of  getting 
the  declared  CPU  time.  Since  the  outputs  are  not  committed  until  that 
time  has  elapsed,  there  are  several  possible  design  patterns  that  can 
be  overlayed  on  this: 

*  Terminator  pattern.  If  a  process  is  not  ready  to  produce  outputs  when 
its  time  has  elapsed,  then  it  has  violated  its  contract  on  declared 
execution  time,  and  it  is  terminated,  not  to  be  invoked  again. 

*  Anytime  pattern.  If  a  process  has  not  completed  its  computation  when 
its  time  has  elapsed,  then  partial  results  are  extracted  and  produced. 

*  Laisez  faire  pattern.  When  the  fire()  method  is  invoked,  it  performs 
a  join  with  the  executing  process,  allowing  it  to  run  to  completion. 

There  are  circumstances  where  each  of  these  patterns  is  appropriate, 
and  the  choice  depends  on  the  application.  It  does  not  make  sense  to 
fix  one  of  these  choices  in  the  modeling  framework. 


Continuous-Time  Domain 

With  the  help  of  Gautam  Biswas  and  others  at  Vanderbilt  University,  Jie 
Liu  identified  limitations  in  the  semantics  of  the  CT  domain  (continuous  time) 
when  combined  with  finite  state  machines  (FSMs)  to 
make  hybrid  systems.  Previously,  actors  were  identified  as  either 
discrete  or  continuous,  and  the  CT  kernel  treated  the  two  differently. 

Any  actor  downstream  from  a  discrete  actor  was  also  discrete.  However, 
when  building  hybrid  systems,  it  is  common  for  an  actor  to  interact 
with  both  discrete  and  continuous  signals.  Jie  solved  the  problem  by 
identifying  signals  instead  of  actors  as  discrete  or  continuous,  and 
using  a  mechanism  similar  to  type  resolution  to  determine  for  each 
signal  whether  it  is  discrete  or  continuous. 
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High  Confidence  Control  of  Multi-Modal  Systems 

In  multi-modal  control  paradigm,  a  set  of  controllers  of  satisfactory 
performance  have  already  been  designed  and  must  be  used.  Each  controller 
may  be  designed  for  a  different  set  of  outputs  in  order  to  meet  the  given 
performance  objectives  and  system  constraints.  When  such  a  collection  of 
control  modes  is  available,  an  important  problem  is  to  be  able  to  accomplish  a 
variety  of  high  level  tasks  by  appropriately  switching  between  the  low-level 
control  modes.  T.  John  Koo,  George  Pappas  (University  of  Pennsylvania),  and 
Shankar  Sastry  presented  a  framework  for  determining  the  sequence  of  control 
modes  that  will  satisfy  reachability  tasks.  Our  framework  exploits  the  structure  of 
output  tracking  controllers  in  order  to  extract  a  finite  graph  where  the  mode 
switching  problem  can  be  efficiently  solved,  and  then  implement  it  using  the 
continuous  controllers.  Our  approach  is  illustrated  on  a  robot  manipulator 
example,  where  we  determine  the  mode 
switching  logic  that  achieves  the  given  reachability  task. 

Hierarchical  System  Architecture  for  Multi-Agent  Multi-Modal  Systems 

T.  John  Koo  presented  a  hierarchical  system  architecture  for  multi-agent  multi¬ 
modal  systems.  The  design  principle  for  the  construction  of  the  hierarchy  is 
based  on  bisimulation  and  therefore  a  higher-level  system  and  a  lower-level 
system  are  bisimilar.  The  layered  system  is  designed  to  promote  proof 
obligations  so  that  system  specification  at  one  level  of  granularity  conforms  with 
system  specification  at  another  level  and  vice  versa.  The  approach  is  illustrated 
on  designing  a  system  architecture  for  executing  a  mission  of  controlling  a  group 
of  autonomous  agents  in  the  pursuit  of  multiple  evaders. 

Hierarchical  Approach  for  Design  of  Multi-Vehicle  Multi-Modal  Embedded 
Software 

Embedded  systems  composed  of  hardware  and  software  components  are 
designed  to  interact  with  a  physical  environment  in  real-time  in  order  to  fulfill 
control  objectives  and  system  specifications.  T.  John  Koo,  Judy  Liebman,  Cedric 
Ma,  and  S.  Shankar  Sastry  addressed  the  complex  design  challenges  in 
embedded  software  by  focusing  on  predictive  and  systematic  hierarchical  design 
methodologies  which  promoted  system  verification  and  validation.  First,  they 
advocate  a  mix  of  top-down,  hierarchical  design  and  bottom-up,  component- 
based  design  for  complex  control  systems.  Second,  it  is  their  point  of  view  that 
at  the  level  closest  to  the  environment  under  control,  the  embedded  software 
needs  to  be  time-triggered  for  guaranteed  safety;  at  the  higher  levels,  we 
advocate  an  asynchronous  hybrid  controller  design.  We  briefly  illustrate  our 
approach  through  an  embedded  software  design  for  the 
control  of  a  group  of  autonomous  vehicles. 

Hardware-in-the-loop  Simulation  of  Multi-Vehicle  Multi-Modal  Embedded 
Systems 

To  design  embedded  systems  for  the  control  of  autonomous  vehicles  for 
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collectively  delivering  high  levels  of  mission  reliability  has  been  putting 
tremendous  pressure  on  control  and  software  designers  in  industry.  Hardware-in- 
the-loop  (HIL)  simulation  allows  testing  an  embedded  computing  device  by 
replacing  the  real-world  environment  at  the  device's  input-output  interface  with  a 
simulated  environment.  HIL  simulation  is  particularly  effective  when  normal 
testing  is  dangerous  or  costly.  HIL  simulation  facilitates  repeatable  testing,  as 
well  as  quantification  of  design  tradeoffs.  T.  John  Koo,  Benjamin  Horowitz,  Judy 
Liebman,  Cedric  Ma,  Ron  Tal,  Tom  Henzinger,  and  Shankar  Sastry  started  to 
look  into  the  issues  on  the  design  of  a  HIL  simulation  system  for  a  group  of 
unmanned  aerial  vehicles  (UAVs). 

Decidable  and  Semi-decidable  Controller  Synthesis  for  Classes  of 
DiscreteTime  Hybrid  Systems 

Rene  Vidal,  Shawn  Schaffert,  Omid  Shakernia,  John  Lygeros,  Shankar  Sastry 
presented  classes  of  discrete  time  hybrid  systems  for  which  the  classical 
algorithm  for  computing  the  maximal  controlled  invariant  set  and  the  least 
restrictive  controller  is  computable  and  guaranteed  to  terminate  in  a  finite  number 
of  iterations.  We  show  how  the  algorithm  can  be  encoded  using  quantifier 
elimination,  which  leads  to  a  semi-decidability  result  for  definable  hybrid  systems. 
For  discrete  time  linear  systems  with  linear  constraints  that  are  either  controllable 
or  nilpotent  and  have  bounded  disturbances,  we  show  that  the  controlled 
invariance  algorithm  terminates  in  a  number  of  iterations  which  is  at  most  the 
dimension  of  the  system.  Both  in  the  hybrid  and  in  the  linear  case,  our  results 
are  much  more  general  than  the  corresponding  ones  for  continuous  time 
systems.  Finally  they  show  that  for  linear  systems  with  ellipsoidal  constraints,  an 
approximated  solution  can  be  obtained  using  robust  convex  programming.  They 
provide  an  example  showing  that  our  algorithm  gives  better  estimations  than 
other  ellipsoidal  methods 

and  is  more  efficient  than  the  exact  method  for  linear  constraints. 
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Chapter  Five:  Accomplishments  in  2003 


Our  effort  integrated  three  tasks:  formal  modeling  and  verification;  embedded 
software  design;  and  controller  design  and  analysis.  We  made  substantial 
progress  in  all  three  directions.  In  addition  in  the  past  year  we  had  a  large 
number  of  successful  hardware  demonstrations  on  our  Berkeley  Aerobotics 
(BEAR)  rotorcraft  UAVs.  Our  group  is  arguably  the  most  advanced  UAV  research 
group  in  the  country  and  the  following  experiments  performed  under  SEC  were 
highlighted  at  the  recent  PI  meeting: 

1 .  Giotto  Implementation  of  Platform  Based  UAV  Controller 

2.  Formation  Flying  with  2  real  and  7  virtual  UAVs. 

3.  UAV  chicken  with  2  UAVs  flying  at  each  other 

4.  Conflict  Resolution  for  multiple  UAVs  (software  demo) 

5.  Dogfights  with  multiple  close  in  UAVs  in  complex  urban  environments 
(software  demo). 

This  was  in  addition  to  complex  acrobatic  maneuvers  (demonstrated  in  the  May 
02  PI  meeting),  vision  based  landing  (presented  by  Northrop  Grumann  as 
possible  transition  target  to  their  FireScout  at  May  03  PI  meeting),  pursuit 
evasion  games.  In  addition  we  showed  some  preliminary  work  on  electric 
rotorcraft  and  low  weight  fixed  wing  UAVs  (the  BAT  UAV). 

The  solution  of  control  problems  for  probabilistic  systems  with  uncertain 
environments. 

Behavioral  properties  of  control  systems  can  be  formalized  as  objectives  in  two- 
player  games.  Turn-based  games  model  asynchronous  interaction  between  the 
players  (the  system  and  its  environment)  by  interleaving  their  moves.  Concurrent 
games  model  synchronous  interaction:  the  players  always  move  simultaneously. 
Probabilistic  moves  can  be  used  to  model  uncertainty  in  the  environment  and 
failure  scenarios  for  the  system.  We  found  efficient  reductions  of  concurrent 
probabilistic  Buchi  and  co-Buchi  games  to  turn-based  games  with  Buchi 
condition  and  parity  winning  condition  with  three  priorities,  respectively. 

Giotto-based  and  platform-based  implementations  of  helicopter  flight 
control  systems 

Giotto  is  a  platform-independent  language  for  specifying  software  for  high- 
performance  control  applications.  We  developed  a  new  approach  to  the 
compilation  of  Giotto.  Following  this  approach,  the  Giotto  compiler  generates 
code  for  a  virtual  machine,  called  the  E  machine,  which  can  be  ported  to  different 
platforms.  The  Giotto  compiler  also  checks  if  the  generated  E  code  is  time  safe 
for  a  given  platform,  that  is,  if  the  platform  offers  sufficient  performance  to  ensure 
that  the  E  machine  semantics.  We  demonstrated  the  feasibility  and  benefits  of 
Giotto-based  control  software  development  by  re-implementing  the  autopilot 
system  of  an  autonomously  flying  model  helicopter.  Giotto  offers  a  clean 
separation  between  the  platform  independent  concerns  of  software  functionality 
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and  I/O  timing,  and  the  platform  dependent  concerns  of  software  scheduling  and 
execution.  Functionality  code  such  as  code  computing  control  laws  can  be 
generated  automatically  from  Simulink  models  or,  as  in  the  case  of  this  project, 
inherited  from  a  legacy  system.  I/O  timing  code  is  generated  automatically  from 
Giotto  models  that  specify  real-time  requirements  such  as  task  frequencies  and 
actuator  update  rates. 

Formation  reconfiguration  planning  for  autonomous  vehicles 

Given  a  group  of  autonomous  vehicles,  an  initial  configuration,  a  final 
configuration,  a  set  of  inter-  and  intra-  vehicle  constraints,  and  a  time  for 
reconfiguration,  the  Formation  Reconfiguration  Planning  problem  is  focused  on 
determining  a  nominal  input  trajectory  for  each  vehicle  such  that  the  group  can 
start  from  the  initial  configuration  and  reach  its  final  configuration  at  the  specified 
time  while  satisfying  the  set  of  inter-  and  intra-  vehicle  constraints.  S.  Zelinski,  T. 
J.  Koo,  and  S.  Sastry  were  interested  in  solving  the  Formation  Reconfiguration 
Planning  problem  for  a  specific  class  of  systems  and  a  particular  form  of  input 
signals  so  that  the  problem  can  be  reformulated  as  an  optimization  problem 
which  can  be  solved  more  efficiently,  especially  for  a  large  group  of  vehicles. 

Design,  Analysis,  and  Implementation  of  the  Embedded  Virtual  Machine 

The  Embedded  Machine  is  a  virtual  machine  that  mediates  in  real  time  the 
interaction  between  software  processes  and  physical  processes.  It  separates  the 
compilation  of  embedded  programs  into  two  phases.  The  first,  platform- 
independent  compiler  phase  generates  E  code  (code  executed  by  the  Embedded 
Machine),  which  supervises  the  timing  -not  the  scheduling-  of  application  tasks 
relative  to  external  events, such  as  clock  ticks  and  sensor  interrupts.  E  code  is 
portable  and  exhibits,  given  an  input  behavior,  predictable  (i.e.,  deterministic) 
timing  and  output  behavior.  The  second,  platform-dependent  compiler 
phase  checks  the  time  safety  of  the  E  code,  that  is,  whether  platform 
performance  (determined  by  the  hardware)  and  platform  utilization  (determined 
by  the  scheduler  of  the  operating  system)  enable  its  timely  execution.  We  have 
used  the  Embedded  Machine  to  compile  and  execute  high-performance  control 
applications  written  in  Giotto,  such  as  the  flight  control  system  of  an  autonomous 
model  helicopter.  For  the  first  time  we  flew  formation  flight  for  9  UAVs  (7  virtual 
and  2  real  UAV)  as  demonstrated  at  the  May  03  PI  meeting. 

Runtime  Platform  for  Timed  Multitasking 

We  checked  in  a  preliminary  runtime  system  for  the  time  multitasking  (TM) 
models  in  Ptolemy  II.  The  runtime  system  is  mainly  a  scheduler  and  some  time 
management;  it  is  written  in  C  and  uses  POSIX  threads.  It  has  been  tested  under 
both  Cygwin  and  Redhat  Linux.  Jie  Liu  (now  at  Parc)  implemented  a  helicopter 
control  example  using  this  runtime  system  and  hand  generated  actor  code.  The 
example  uses  the  hardware-in-the-loop  simulator  that  Judy  Liebman,  Cedric  Ma, 
and  Ben  Horowitz  used  to  demonstrate  Giotto.  Jie  took  the  same  helicopter, 
sensor,  and  actuator  code  from  them,  and  replaced  the  Giotto  controller  with  TM 
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controller.  The  TM  model  effectively  reduced  the  one-sample  delay  per  actor 
introduced  by  Giotto. 

Nonlinear  Model  Predictive  Planning  and  Control 

H.  Jin  Kim  presented  nonlinear  model  predictive  planning  &  control  framework 
which  combines  trajectory  planning  and  control  into  a  single  problem,  using  ideas 
from  potential-field  based  navigation  for  real-time  path  planning  and  nonlinear 
model  predictive  control  for  optimal  control  of  nonlinear  multi-input,  multi-output 
systems  with  input/state  constraints.  She  incorporates  a  tracking  performance, 
potential  function,  state  constraints  into  the  cost  function  to  minimize,  and  use 
gradient-descent  for  on-line  optimization.  A  scenario  with  five  UAVs  which  are 
given  straight  line  trajectories  that  will  lead  to  collision  is  considered.  By  using  the 
framework,  the  safe  trajectory  of  the  vehicles  are  dynamically  replanned  and 
tracked  by  the  vehicles  under  input  saturation  and  state  constraints. 

We  used  this  work  to  demonstrate  several  important  experiments: 

1 .  UAV  chicken  with  2  UAVs  flying  at  each  other  (hardware  experiment 
demonstrated  at  PI  Meeting  May  03) 

2.  Conflict  Resolution  for  multiple  UAVs  (software  demo  showed  in  May  03) 
and  also  to  be  used  in  the  Softwalls  Project. 

3.  Dogfights  with  multiple  close  in  UAVs  in  complex  urban  environments 
(software  demo  given  in  May  03). 

Additionally  continuing  in  the  style  of  the  reports  of  the  previous  years  our 
detailed  accomplishments  were  as  follows: 

Time-safety  checking  for  embedded  programs 

Giotto  is  a  platform-independent  language  for  specifying  software  for  high- 
performance  control  applications.  We  developed  a  new  approach  to 
the  compilation  of  Giotto.  Following  this  approach,  the  Giotto  compiler  generates 
code  for  a  virtual  machine,  called  the  E  machine, 

which  can  be  ported  to  different  platforms.  The  Giotto  compiler  also  checks  if  the 
generated  E  code  is  time  safe  for  a  given  platform,  that 
is,  if  the  platform  offers  sufficient  performance  to  ensure  that  the  E  code  is 
executed  in  a  timely  fashion  that  conforms  with  the  Giotto  semantics.  Time- 
safety  checking  requires  a  schedulability  analysis.  We  showed  that  while  for 
arbitrary  E  code,  the  analysis  is  exponential,  for  E  code  generated  from  typical 
Giotto  programs,  the  analysis  is  polynomial.  This  supports  our  claim  that  Giotto 
identifies  a  useful  fragment  of  embedded  programs. 

Trading  probability  for  fairness 

Behavioral  properties  of  control  systems  can  be  formalized  as  objectives  in  two- 
player  games.  Turn-based  games  model  asynchronous  interaction  between  the 
players  (the  system  and  its  environment)  by  interleaving  their  moves.  Concurrent 
games  model  synchronous  interaction:  the  players  always  move  simultaneously. 
Probabilistic  moves  can  be  used  to  model  uncertainty  in  the  environment  and 
failure  scenarios  for  the  system.  Infinitary  winning  criteria  (control  objectives)  are 
considered:  Buchi,  co-Buchi,  and  more  general  parity  conditions.  A 
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generalization  of  determinacy  for  parity  games  to  concurrent  parity  games 
demands  probabilistic  (mixed)  strategies:  either  player  1  has  a  mixed  strategy  to 
win  with  probability  1  almost-sure  winning),  or  player  2  has  a  mixed  strategy  to 
win  with  positive  probability. 

We  found  efficient  reductions  of  concurrent  probabilistic  Buchi  and 
co-Buchi  games  to  turn-based  games  with  Buchi  condition  and  parity 
winning  condition  with  three  priorities,  respectively.  From  a  theoretical  point  of 
view,  the  latter  reduction  shows  that  one  can  trade  the  probabilistic  nature  of 
almost-sure  winning  for  a  more  general  parity  (fairness)  condition.  The 
reductions  improve  understanding  of  concurrent  games  and  provide  an 
alternative  simple  proof  of  determinacy  of  concurrent  Buchi  and  co-Buchi  games. 
From  a  practical  point  of  view,  the  reductions  turn  solvers  of  turn-based  games 
into  solvers  of  concurrent  probabilistic  games.  Thus  improvements  in  the  well- 
studied  algorithms  for  the  former  carry  over  immediately  to  the  latter.  In 
particular,  a  recent  improvement  in  the  complexity  of  solving  turn-based  parity 
games  yields  an  improvement  in  time  complexity  of  solving  concurrent 
probabilistic  co-Buchi  games  from  cubic  to  quadratic. 

The  Element  of  Surprise  in  Timed  Games 

The  control  of  real-time  and  hybrid  systems  is  naturally  formulated  and  solved  as 
timed  games  (controller  versus  plant).  We  developed  a  symmetric  formulation  of 
such  timed  games  and  provided  the  corresponding  control  algorithms. 

We  considered  concurrent  two-person  games  played  in  real  time,  in  which  the 
players  decide  both  which  action  to  play,  and  when  to  play  it. 

Such  timed  games  differ  from  untimed  games  in  two  essential  ways. 

First,  players  can  take  each  other  by  surprise,  because  actions  are  played  with 
delays  that  cannot  be  anticipated  by  the  opponent. 

Second,  a  player  should  not  be  able  to  win  the  game  by  preventing  time  from 
diverging.  We  presented  a  model  of  timed  games  that  preserves  the  element  of 
surprise  and  accounts  for  time  divergence  in  a  way  that  treats  both  players 
symmetrically  and  applies  to  all  omega-regular  winning  conditions. 

We  proved  that  the  ability  to  take  each  other  by  surprise  adds  extra  power  to  the 
players.  For  the  case  that  the  games  are  specified  in  the  style  of  timed 
automata,  we  provided  symbolic  algorithms  for  their  solution  with  respect  to  all 
omega-regular  winning  conditions.  We  also  showed  that  for  these  timed  games, 
memory  strategies  are  more  powerful  than  memoryless  strategies  already  in  the 
case  of  reachability  objectives. 

Schedule  Carrying  Code 

To  guarantee  the  correct  execution  of  a  hard  real-time  program  on  a 
given  platform,  the  scheduler  must  ensure  that  all  deadlines  are  met. 

We  introduced  the  paradigm  of  schedule-carrying  code  (SCC),  where  the 
compiler  proves  the  existence  of  a  feasible  schedule  by  generating 
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such  a  schedule,  which  is  then  attached  to  the  generated  code  in  the 
form  of  executable  instructions  that  remove  the  need  for  a  system 
scheduler.  In  this  way,  the  schedule  is  produced  once,  and 
revalidated  and  executed  with  each  use.  We  evaluated  SCC  both  in 
theory  and  practice.  In  theory,  we  gave  two  scenarios,  of 
nonpreemptive  and  distributed  scheduling  for  Giotto  programs,  where 
the  generation  of  a  feasible  schedule  is  hard,  while  the  validation  of 
scheduling  instructions  that  are  attached  to  the  code  is  easy.  In 
practice,  we  implemented  SCC  and  show  that  explicit  scheduling 
instructions  can  reduce  the  scheduling  overhead  up  to  35  percent,  and 
can  provide  an  efficient,  flexible,  and  verifiable  means  for  compiling 
Giotto  on  complex  architectures,  such  as  the  TTA. 

Causality  in  Mixed-Signal  and  Hybrid  Systems 

Together  with  Jie  Liu  of  PARC,  we  have  completed  a  study  of  the  semantics  of 
causality  in  mixed-signal  and  hybrid  models  [6],  This  study  extends  the 
application  of  the  Cantor  metric  as  a  mathematical  tool  for  defining  causality  from 
purely  discrete  models  to  mixed-signal  and  hybrid  models.  Using  the  Cantor 
metric,  which  represents  timed  signals,  continuous  or  discrete,  in  a  metric  space, 
we  define  causality  as  contractive  properties  of  processes  operating  on  these 
signals. 

Thus,  the  Banach  fixed  point  theorem  can  be  applied  to  establish  conditions  for 
the  existence,  uniqueness,  and  liveness  of  the  behaviors  for  mixed-signal  and 
hybrid  systems.  The  results  also  provide  theoretical  foundations  for  the 
simulation  technologies  for  such  systems,  including  the  time-marching  strategy, 
evaluation  of  feedback  loops,  and  the  necessity  of  supporting  rollback. 

A  Giotto-based  helicopter  control  system 

We  demonstrated  the  feasibility  and  benefits  of  Giotto-based  control  software 
development  by  reimplementing  the  autopilot  system  of  an  autonomously  flying 
model  helicopter.  Giotto  offers  a  clean  separation  between  the  platform 
independent  concerns  of  software  functionality  and  I/O  timing,  and  the  platform 
dependent  concerns  of  software  scheduling  and  execution.  Functionality  code 
such  as  code  computing  control  laws  can  be  generated  automatically  from 
Simulink  models  or,  as  in  the  case  of  this  project,  inherited  from  a  legacy  system. 
I/O  timing  code  is  generated  automatically  from  Giotto  models  that  specify  real¬ 
time  requirements  such  as  task  frequencies  and  actuator  update  rates.  We 
extend  Simulink  to  support  the  design  of  Giotto  models,  and  from  these  models, 
the  automatic  generation  of  Giotto  code  that  supervises  the  interaction  of  the 
functionality  code  with  the  physical  environment.  The  Giotto  compiler  performs  a 
schedulability  analysis  on  the  Giotto  code,  and  generates  timing  code  for  the 
helicopter  platform.  The  Giotto  methodology  guarantees  the  stringent  hard  real¬ 
time  requirements  of  the  autopilot  system,  and  at  the  same  time  supports  the 
automation  of  the  software  development  process  in  a  way  that  produces  a 
transparent  software  architecture  with  predictable  behavior  and  reusable 
components. 
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Caltech  Platform 

We  continue  to  work  with  Caltech  on  their  rolling  platform  driven  by 
two  ducted  fans.  In  particular,  Steve  Neuendorfer  continues  to  develop 
code  generation  with  the  objective  of  providing  a  prototyping  software 
environment  for  the  platform.  Caltech  has  debugged  the  PIC  code  for  the 
platform,  and  has  provided  us  with  an  instance  of  the  board,  but  there 
has  not  been  time  yet  to  do  much  with  it. 

A  Ptolemy  II  model  of  2D  multi-modal  helicopter  example 

John  Koo  contributed  a  2-D  multi-modal  helicopter  example  to  the 
Ptolemy  II  demo  library.  The  model  is  based  on  T.  J.  Koo,  S.  Sastry 
"Output  Tracking  Control  Design  of  a  Helicopter  Model  Based  on 
Approximate  Linearization,"  In  Proceedings  of  IEEE  Conference  on 
Decision  and  Control,  Florida,  December  1998.  The  control  sequence  is  derived 
based  on  T.  J.  Koo,  G.  J.  Pappas,  and  S.  Sastry,  "Mode 
Switching  Synthesis  for  Reachability  Specifications,"  Hybrid  Systems: 
Computation  and  Control,  M.  D.  Di  Benedetto  and  A.  Sangiovanni-Vincentelli 
(Eds.),  Lecture  Notes  in  Computer  Science,  Vol.  2034,  pp.  333-346,  Springer- 
Verlag,  2001. 

Mobile  Models  and  the  Caltech  Vehicle 

Steve  Neuendorffer  has  demonstrated  the  use  of  Ptolemy  II  for  designing 
controllers  for  the  Caltech  ducted  fan  vehicle.  Four  configurations 
have  been  shown: 

(1 )  the  controller  and  vehicle  are  both  simulated; 

(2)  the  controller  is  code-generated  and  the  generated  code  interacts 
with  a  simulation  of  the  vehicle  (hardware  in  the  loop  simulation); 

(3)  the  controller  is  simulated  and  interacts  with  the  physical  vehicle 
(simulation-based  rapid  prototyping);  and 

(4)  the  controller  is  code-generated  and  the  generated  code  interacts 
with  the  physical  vehicle  (deployment). 

To  facilitate  experimentation  with  control  algorithms,  Yang  Zhao 
developed  a  MobileModel  Ptolemy  II  actor,  which  is  a  higher-order  actor 
that  accepts  a  model  definition  at  one  of  its  input  ports,  and  then 
executes  that  model  to  process  data  provided  at  the  other  input  ports. 

The  prototype  application  of  this  concept  uses  a  Ptolemy  II  model  on 
the  vehicle  with  the  MobileModel  actor  implementing  the  control  laws, 
and  a  separate  supervisory  model  on  another  laptop  providing  the 
control  laws  over  a  CORBA  channel  (using  a  wireless  network). 

A  key  issue  for  such  mobile  models  is  security.  Currently,  the 
prototype  accepts  models  provided  in  cleartext  XML  and  makes  no 
distinction  between  models  that  use  actors  that  convey  authority  (e.g. 
actors  that  read  or  write  local  files)  and  models  with  no  such  actors. 


29 


An  essential  next  step  is  to  develop  a  security  policy  for  mobile  models 
that  enables  secure  (encrypted)  transport  of  digitally  signed 
executable  models  and  regulated  granting  of  authority  to  such  models. 

Hierarchical  Hybrid  Systems  Models 

With  support  from  the  Mobies  program  and  from  NSF,  the  Ptolemy  group  at 
Berkeley  developed  a  "hybrid  systems  visual  modeler"  called  HyVisual.  HyVisual 
is  based  on  Ptolemy  II.  As  part  of  the  SEC  effort,  HyVisual 
has  been  used  to  construct  various  hybrid  systems  models,  and  a  key  semantic 
issue  has  been  identified  (by  John  Koo). 

HyVisual  starts  with  Simulink-like  block-diagrams  that  represent  ordinary 

differential  equations.  Unlike  Simulink,  it  does  not  attempt 

to  solve  algebraic  equations,  but  rather  requires  that  any  directed 

loop  in  the  model  include  at  least  one  integrator.  This  is  sufficient 

to  ensure  that  the  execution  trace  is  uniquely  defined,  at  least  denotationally  (that 

is,  that  the  denotational  semantics  is  determinate,  see  [6]).  HyVisual  augments 

ODEs  with  modal  models,  where 

a  finite-state  machine  governs  discrete  switching  between  modes,  and 
each  mode  represents  a  subsystem  (called  the  "refinement"  of  the  mode).  The 
refinement  can  be  a  continuous-time  model,  a  discrete  model,  or  a  memoryless 
computation;  the  information  hiding  software  architecture  of 
Ptolemy  II  ensures  consistent  behavior  across  such  heterogeneous 
models.  However,  this  same  information  hiding  prevents  the  solver  at 
one  level  of  the  hierarchy  from  knowing  the  causality  properties  of 
another  level  of  the  hierarchy. 

We  have  begun  working  on  an  abstract  representation  of  causality 
properties  that  can  be  exposed  as  an  interface  definition  across  levels 
of  the  hierarchy,  and  that  can  be  composed  so  that  in  a  modal  model 
with  multiple  refinements,  the  interfaces  of  the  refinements  can  be 
merged  to  define  an  interface  of  the  modal  model.  This  method  will 
apply  not  only  to  the  continuous-time  ODE-based  modeling  of  HyVisual, 
but  also  to  discrete-event  modeling,  synchronous/reactive  modeling,  and 
constraint-based  models  of  computation. 

More  information  about  HyVisual  can  be  found  at: 

http://ptolemy.eecs.berkeley.edu/hyvisual/ 

Localization  Technology 

David  Lee  and  Paul  Yang  completed  their  Ptolemy  II  implementation  of  a 
video-based  localization  technique  for  the  Caltech  ducted  fan  vehicle. 

This  localization  program  uses  a  webcam  and  JMF  (the  Java  Media 
Framework).  It  uses  color  segmentation  to  track  the  vehicle.  Two 
separate  color  dots  are  placed  on  the  cart,  with  one  color  used  for 
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position  and  a  different  color  for  rotation. 

During  this  reporting  period,  David  and  Paul  implemented: 

1)  Automatic  Color  Calibration  using  a  histogram. 

2)  Distance  calculations  based  on  pixel  information. 

Platform-Based  Embedded  Software  Design  and  System  Integration  for 
Autonomous  Vehicles 

Automatic  control  systems  typically  incorporate  legacy  code  and 
components  that  were  originally  designed  to  operate  independently. 

Furthermore,  they  operate  under  stringent  safety  and  timing  constraints.  Current 
design  strategies  deal  with  these  requirements 

and  characteristics  with  ad  hoc  approaches.  In  particular,  when  designing 
control  laws,  implementation  constraints  are  often  ignored  or  cursorily  estimated. 
Indeed,  costly  redesigns  are  needed  after  a  prototype  of  the  control  system  is 
built  due  to  missed  timing  constraints  and  subtle  transient  errors.  T.  J.  Koo,  J. 
Liebman,  C.  Ma, 

B.  Horowitz,  A.  Sangiovanni-Vincentelli,  and  S.  Sastry  used  the  concepts  of 
platform-based  design  to  develop  a  methodology  for  the  design  of  automatic 
control  systems  that  built  in  modularity  and 

correct-by-construction  procedures.  We  illustrated  our  strategy  by  describing  the 
(successful)  application  of  the  methodology  to  the  design  of  a  time-based  control 
system  for  a  helicopter-based 

Uninhabited  Aerial  Vehicle  (UAV)  so  the  the  following  objectives  can  be  met: 

a.  The  use  of  platform-based  design  allows  us  to  build  a  bridge 
between  the  time-based  controller  application  and  the  non-time-based 
sensors  and  actuators. 

b.  A  time-based  controller  eliminates  the  timing  irregularities 
present  in  first  generation  system.  Further,  the  Giotto  compiler 
ensures  that  the  controller  application  meets  its  timing  requirements. 

c.  Our  platform-based  design  achieves  a  high  degree  of 
modularity.  For  example,  to  substitute  a  different  sensor  suite  in  our 
first  redesign  requires  only  changes  to  the  data  processor  and  the 
data  formatting  library.  The  data  processor  would  require  a  different 
sensor  initialization  routine  and  a  new  circular  buffer;  the 
formatting  library  would  need  a  new  format  conversion 

routine.  However,  no  part  of  the  controller  application  would  need  to 
be  changed. 

Optimization-based  Formation  Reconfiguration  Planning  For  Autonomous 
Vehicles 

Given  a  group  of  autonomous  vehicles,  an  initial  configuration,  a  final 
configuration,  a  set  of  inter-  and  intra-  vehicle  constraints,  and  a  time  for 
reconfiguration,  the  Formation  Reconfiguration  Planning  problem  is  focused  on 
determining  a  nominal  input  trajectory  for  each  vehicle  such  that  the  group  can 
start  from  the  initial  configuration 
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and  reach  its  final  configuration  at  the  specified  time  while  satisfying  the  set  of 
inter-  and  intra-  vehicle  constraints.  S.  Zelinski,  T.  J.  Koo,  and  S.  Sastry  were 
interested  in  solving  the  Formation  Reconfiguration  Planning  problem  for  a 
specific  class  of  systems  and  a  particular  form  of  input  signals  so  that  the 
problem  can  be  reformulated  as  an  optimization  problem  which  can  be  solved 
more  efficiently,  especially  for  a  large  group  of  vehicles.  Optimization  has  proved 
to  be  a  successful  solution  to  the  FRP  problem.  Our  method  of  implementation  is 
general  and  portable  allowing  for  use  in  a  wide  range  of  applications  for 
coordinated  robots.  For  example,  our  method  could  easily  be  transported  to  two 
dimensions  for  ground  robot  coordination.  This  centralized  control  scheme  has 
limitations  in  applications  where  formations  are  very  large  or  communication  is 
disrupted.  For  such  applications,  a  decentralized  control  scheme  is  preferred.  We 
are  currently  working  on  a  decentralized  approach  to  the  FRP  problem  where 
each  vehicle  produces  its  own  localized  solution  based  on  only  local  sensor 
information  about  its  neighboring  vehicles.  As  expected,  this  is  proving  to  be  a 
more  complex  problem.  Therefore,  centralized  control  is  preferred  in  applications 
for  smaller  fully  connected  formations. 

Formation  Reconfiguration  For  Autonomous  Vehicles 

Coordination  of  multiple  unmanned  aerial  vehicles  (UAVs)  poses  significant 
theoretical  and  technical  challenges.  Recent  advances  in 
sensing,  communication  and  computation  enable  the  conduct  of 
cooperative  multiple-UAV  mission  deemed  impossible  in  the  recent  past. 

T.  John  Koo,  Shannon  Zelinski  and  Shankar  Sastry  worked  on  solving  the 
Formation  Reconfiguration  Planning  (FRP)  problem  which  is  focused  on 
determining  a  nominal  state  and  input  trajectory  for  each  vehicle  such 
that  the  group  can  start  from  the  given  initial  configuration  and  reach 
its  given  final  configuration  at  the  specified  time  while  satisfying  a 
set  of  given  inter-  and  intra-  vehicle  constraints.  Each  solution  of  a 
FRP  problem  represents  a  distinct  reconfiguration  mode.  When  coupled 
with  formation  keeping  modes,  they  can  form  a  hybrid  automaton  of 
formation  maneuvers  in  which  a  transition  from  one  formation  maneuver 
to  another  formation  maneuver  is  governed  by  a  finite  automaton.  We 
show  a  simulation  results  of  a  group  of  9  UAVs  performing  a  sequence  of 
formation  reconfigurations. 

Collision  Avoidance  based  on  Nonlinear  Model  Predictive  Control 

H.  Jin  Kim  presented  nonlinear  model  predictive  planning  and  control 
framework  which  combines  trajectory  planning  and  control  into  a  single 
problem,  using  ideas  from  potential-field  based  navigation  for  real-time  path 
planning  and  nonlinear  model  predictive  control  for 
optimal  control  of  nonlinear  multi-input,  multi-output  systems  with 
input/state  constraints.  She  incorporates  a  tracking  performance, 
potential  function,  state  constraints  into  the  cost  function  to 
minimize,  and  use  gradient-descent  for  on-line  optimization.  A  scenario 
with  three  UAVs  which  are  given  straight  line  trajectories  that  will 
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lead  to  collision  is  considered.  By  using  the  framework,  the  safe 
trajectory  of  the  vehicles  are  dynamically  replanned  and  tracked  by  the 
vehicles  under  input  saturation  and  state  constraints. 

Platform-Based  Embedded  Software  Development 

B.  Horowitz  used  the  concepts  of  platform-based  design  to  develop  a 
methodology  for  the  design  of  automatic  control  systems  that  built  in  modularity 
and  correct-by-construction  procedures.  The  use  of  platform-based  design  allows 
us  to  build  a  bridge  between  the  time-based  controller  application  and  the  non¬ 
time-based  sensors  and  actuators.  A  time-based  controller  eliminates  the  timing 
irregularities  present  in  first  generation  system.  Further,  the  Giotto  compiler 
ensures  that  the  controller  application  meets  its  timing  requirements.  Our 
platform-based  design  achieves  a  high  degree  of  modularity. 

Software  Releases  and  Technology  Transfer 

•  A  Simulink  model  of  a  helicopter  control  system,  including  vehicle 
dynamics,  controllers,  sensor  models,  and  Kalman  filter,  was  distributed  to 
Northrop  Grumann  for  the  development  of  High  Confidence  Control 
Design  for  UAVs. 

•  Code  for  vision  based  landing  on  UAVs  for  possible  use  in  a  FireScout 
landing  demo  was  also  distributed  to  Northrop  Grumann. 

•  We  released  Ptolemy  II  2.0.1  and  Ptplot  5.2.  For  details,  see: 
http://ptolemy.eecs.berkeley.edu 

•  Versions  of  Giotto  and  the  E  Machine  were  released.  For  Details  see 
http://sec.eecs.berkeley.edu 

•  Conflict  Resolution  methods  for  groups  of  UAVs  and  unmanned  vehicles 
are  being  considered  under  the  Softwall  project  for  integration  by 
Honeywell,  Inc.  and  Boeing  in  their  commercial  aircraft  companies. 

•  We  helped  Boeing  script  a  new  scenario  for  use  in  the  Final  Capstone 
Demo. 
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Chapter  Six:  Accomplishments  in  2004 


There  was  a  fair  amount  that  we  accomplished  along  the  three  lines  of  research 
on  the  project:  formal  modeling  and  verification;  embedded  software  design;  and 
controller  design  and  analysis.  We  will  discuss  these  here.  We  are  also 
interested  in  participating  on  the  Boeing  team  in  the  Final  Capstone  Demo  using 
our  vision  based  and  model  predictive  controller  based  scenario  for  collaborative 
and  non-collaborative  operations  of  UAVs. 

Formal  Modeling  and  Verification 

We  performed  the  following  tasks  in  03-04: 

•  Implement  and  evaluate  E  (embedded)  and  S  (scheduling)  machine  on 
various  processors  without  real-time  operating  system  as  intermediary 
(December  2003-June  2004) 

•  Develop  and  implement  xGiotto  (Extended  Giotto)  which  permits  reaction 
to  asynchronous  events  in  addition  to  time  triggered  communication 
(March  2004) 

•  Formulate  a  discounted  version  of  temporal  logics  and  solve  the 
corresponding  model  checking  problems  to  provide  algorithms  for 
verifying  robust  probabilistic  hybrid  systems.  (June  2004). 

Embedded  Software  Design 

•  Implement  the  embedded  machine  on  KURT  Linux,  and  adapt  KURT 
Linux  to  provide  us  with  an  extensible  and  adaptable  RTOS  infrastructure. 
(Feb  2004) 

•  Retarget  Ptolemy  II  code  generation  to  produce  code  that  runs  under 
KURT  Linux,  including  code  generated  from  Giotto,  with  the  FSM  domain 
providing  the  mode  changes  and  SDF  providing  the  task  definitions. 
(November  2003) 

•  Develop  an  interface  theory  for  causality  properties  and  apply  it  to 
facilitate  hierarchical  hybrid  system  modeling.  (May  2004) 

•  Develop  a  security  policy  for  mobile  models  that  enables  secure 
(encrypted)  transport  of  digitally  signed  executable  models  and  regulated 
granting  of  authority  to  such  models.  (May  2004) 

•  Develop  CORBA  and  OCP  based  mobile  model  mechanisms  for 
distributed  control  problems.  (December  2003). 
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Control  Systems  Design 

•  Implementation  of  platform  based  controller  for  formation  flying  and 
formation  change  maneuvers  using  a  combination  of  time  triggered  and 
asynchronous  controllers  (August  2003) 

•  Non-cooperative  operations  of  UAVs  using  provably  correct  model 
predictive  algorithms  (November  2003) 

•  Navigation  in  cluttered  urban  environments  without  being  stuck  in  the  local 
minima  of  potential  field  based  algorithms.  (February  2004) 

Discounting  the  Future  in  Systems  Theory 

Discounting  the  future  means  that  the  value,  today,  of  a  unit  payoff  is 
1  if  the  payoff  occurs  today,  a  if  it  occurs  tomorrow,  aA2  if  it  occurs 
the  day  after  tomorrow,  and  so  on,  for  some  real-valued  discount  factor 
0  <  a  <  1 .  Discounting  (or  inflation)  is  a  key  paradigm  in  economics  and  has  been 
studied  in  Markov  decision  processes  as  well  as  game  theory.  We  submit  that 
discounting  also  has  a  natural  place  in  systems  engineering:  for  nonterminating 
systems,  a  potential  bug  in  the  far-away  future  is  less  troubling  than  a  potential 
bug  today.  We  therefore  developed  a  systems  theory  with  discounting.  Our 
theory  includes  several  basic  elements:  discounted  versions  of  system  properties 
that  correspond  to  the  omega-regular  properties,  fixpoint-based  algorithms  for 
checking  discounted  properties,  and  a  quantitative  notion  of  bisimilarity  for 
capturing  the  difference  between  two  states  with  respect  to  discounted 
properties.  We  developed  the  theory  in  a  general  form  that  applies  to  probabilistic 
systems  as  well  as  multicomponent  systems  (games),  but  it  readily  specializes  to 
classical  transition  systems.  We  showed  that  discounting,  besides  its  natural 
practical  appeal,  has  also  several  mathematical  benefits.  First,  the  resulting 
theory  is  robust,  in  that  small  perturbations  of  a  system  can  cause  only  small 
changes  in  the  properties  of  the  system.  This  is  of  particular  relevance  for  hybrid 
systems,  where  the  classical  models  are  not  robust.  Second,  the  theory  is 
computational,  in  that  the  values  of  discounted  properties,  as  well  as  the 
discounted  bisimilarity 

distance  between  states,  can  be  computed  to  any  desired  degree  of  precision. 

Counter  example-guided  Control 

A  major  hurdle  in  the  algorithmic  verification  and  control  of  system  is  the  need  to 
find  suitable  abstract  models,  which  omit  enough  details 
to  overcome  the  state-explosion  problem,  but  retain  enough  details  to  exhibit 
satisfaction  or  controllability  with  respect  to  the  specification.  The  paradigm  of 
counterexample-guided  abstraction  refinement  suggests  a  fully  automatic  way  of 
finding  suitable  abstract  models:  one  starts  with  a  coarse  abstraction,  attempts  to 
verify  or  control  the  abstract  model,  and  if  this  attempt  fails  and  the  abstract 
counterexample  does  not  correspond  to  a  concrete  counterexample,  then 
one  uses  the  spurious  counterexample  to  guide  the  refinement  of  the 
abstract  model.  We  developed  a  counterexample-guided  refinement  algorithm  for 
solving  omega-regular  control  objectives.  The  main  difficulty  is  that  in  control, 
unlike  in  verification,  counterexamples  are  strategies  in  a  game  between  system 
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and  controller.  In  the  case  the  controller  has  no  choices,  our  scheme  subsumes 
known  counterexample-guided  refinement  algorithms  for  the  verification  of 
omega-regular  specifications.  Our  algorithm  is  useful  in  all  situations  where 
omega-regular  games  need  to  be  solved,  such  as  supervisory  control,  sequential 
and  program  synthesis,  and  modular  verification.  The  algorithm  is  fully  symbolic, 
and  therefore  applicable  also  to 
infinite-state  systems. 


Stack-size  Analysis  for  Interrupt-driven  Programs 

We  solved  the  problems  of  determining  stack  boundedness  and  the  exact 
maximum  stack  size  for  three  classes  of  interrupt-driven  programs. 
Interrupt-driven  programs  are  used  in  many  real-time  applications  that 
require  responsive  interrupt  handling.  In  order  to  ensure  responsiveness, 
programmers  often  enable  interrupt  processing  in  the  body  of  lower-priority 
interrupt  handlers.  In  such  programs  a  programming  error  can  allow  interrupt 
handlers  to  be  interrupted  in  cyclic  fashion  to  lead  to  an  unbounded  stack, 
causing  the  system  to  crash.  For  a  restricted  class  of  interrupt-driven  programs, 
we  showed  that  there  is  a  polynomial-time  procedure  to  check  stack 
boundedness,  while  determining  the  exact  maximum  stack  size  is  PSPACE- 
complete.  For  a  larger  class  of  programs,  the  two  problems  are  both  PSPACE- 
complete,  and  for  the  largest  class  of  programs  we  considered,  the  two  problems 
are  PSPACE-hard  and  can  be  solved  in  exponential  time. 

Giotto  and  the  E  Machine 

Steve  Neuendorffer,  Marco  Sanvido,  Christoph  Kirsch  were  able  to 
automatically  generate  code  from  a  Giotto  model  in  Ptolemy,  and  target 
the  embedded  machine.  Task  code  is  generated  from  SDF  models,  E  machine 
drivers  are  generated  from  the  connectivity  in  the  model,  and  the  Giotto  compiler 
is  used  to  generate  the  E  machine  code.  This  heavily  leveraged  Haiyang  Zheng's 
previous  work  generating  Giotto  code  from  Ptolemy.  Essentially  they  filled  in  the 
drivers  and  task  code  using  code  generated  using  Copernicus.  Currently,  only 
single  mode  systemsare  supported,  and  the  Giotto  model  is  assumed  to  be  at  the 
top  level.  This  work  was  cooperative  with  Mobies. 

Open  Control  Platform 

Steve  Neuendorffer  and  Yang  Zhao  attended  the  OCP  workshop  in  January 
at  Caltech.  They  tested  some  of  the  concepts  that  we  have  been  working 
on,  and  came  away  with  a  few  observations.  Steve  Neuendorffer  tested  his 
"crazyboard"  controller  models,  built  in  Ptolemy  II,  on  the  Caltech  hardware. 

There  was  one  laptop  sitting  on  the  Crazyboard  and  running  a  very  simple  model 
to  receive  control  commands,  and  there  was  another  laptop  that  runs  the  control 
algorithm  remotely  and  sends  commands  to  the  laptop  on  the  Crazyboard.  The 
reason  for  two  laptops  is  that  they  wanted  to  test  different  control  algorithms  and 
modify  algorithms  conveniently.  But  this  approach  has  a  drawback:  the  latency  is 
quite  big  due  to  the  control  being  over  the  network. 
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Yang  and  Steve  have  suggested  an  alternative  architecture  where  the  control  law 
is  implemented  in  a  migrating  model  that  is  transported  over  the  network  and 
executed  locally.  Yang  built  an  experimental  setup  where  a  MobileModel  actor 
accepts  a  model  over  the  network  and  executes  a  model  to  process  streaming 
inputs.  The  model  can  be  updated  in  real  time  by  a  supervisory  model  on  the 
second  laptop,  but  the  control  loop  no  longer  includes  the  network,  thus  greatly 
improving  performance.  This  work  continues,  with  the  objective  of  building  an 
infrastructure  that  is  easy  for  control  systems  designers  to  use. 

Yang  has  also  experimented  with  using  JXTA  to  discover  control  services 
and  Crazyboard  configurations.  JXTA  is  a  discovery  infrastructure  from  Sun 
Microsystems.  A  key  observation  that  Yang  and  Steve  made  is  that  there  is  a 
need  for  a  modeling  and  simulation  environment  for  developers  designing 
distributed  systems  based  on  OCP.  Designers  today  can  use  Simulink  to  draw 
the  component  diagram  of  a  system,  but  they  cannot  do  the  simulation  in 
Simulink  since  it  does  not  provide  the  proper  semantics.  It  would  be  possible  to 
build  a  Ptolemy  II  domain  with  OCP  semantics  that  might  serve  this  purpose. 

Nonlinear  Model  Predictive  Planning  and  Control 

H.  Jin  Kim  presented  nonlinear  model  predictive  planning  &  control  framework 
which  combines  trajectory  planning  and  control  into  a  single  problem,  using  ideas 
from  potential-field  based  navigation  for  real-time  path  planning  and  nonlinear 
model  predictive  control  for  optimal  control  of  nonlinear  multi-input,  multi-output 
systems  with  input/state  constraints.  She  incorporates  a  tracking  performance, 
potential  function,  state  constraints  into  the  cost  function  to  minimize,  and  use 
gradient-descent  for  on-line  optimization.  A  scenario  with  three  UAVs  which  are 
given  straight  line  trajectories  that  will  lead  to  collision  is  considered.  By  using  the 
framework,  the  safe  trajectory  of  the  vehicles  are  dynamically  replanned  and 
tracked  by  the  vehicles  under  input  saturation  and  state  constraints. 

Optimization-based  Formation  Reconfiguration  Planning  For  Autonomous 
Vehicles 

Given  a  group  of  autonomous  vehicles,  an  initial  configuration,  a  final 
configuration,  a  set  of  inter-  and  intra-vehicle  constraints,  and  a 
time  for  reconfiguration,  the  Formation  Reconfiguration  Planning  problem  is 
focused  on  determining  a  nominal  input  trajectory  for  each  vehicle  such  that  the 
group  can  start  from  the  initial  configuration  and  reach  its  final  configuration  at 
the  specified  time  while  satisfying  the  set  of  inter-  and  intra-  vehicle  constraints. 

S.  Zelinski,  T.  J.  Koo,  and  S.  Sastry  were  interested  in  solving  the 
Formation  Reconfiguration  Planning  problem  for  a  specific  class  of 
systems  and  a  particular  form  of  input  signals  so  that  the  problem  can 
be  reformulated  as  an  optimization  problem  which  can  be  solved  more 
efficiently,  especially  for  a  large  group  of  vehicles.  Optimization  has 
proven  to  be  a  successful  solution  to  the  FRP  problem.  Our  method  of 
implementation  is  general  and  portable  allowing  for  use  in  a  wide  range  of 
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applications  for  coordinated  robots.  For  example,  our  method  could  easily  be 
transported  to  two  dimensions  for  ground  robot  coordination.  This  centralized 
control  scheme  has  limitations  in  applications  where  formations  are  very  large  or 
communication  is  disrupted.  For  such  applications,  a  decentralized  control 
scheme  is  preferred.  We  are  currently  working  on  a  decentralized  approach  to 
the  FRP  problem  where  each  vehicle  produces  its  own  localized  solution  based 
on  only  information  about  its  neighboring  vehicles.  As  expected,  this  is  proving 
to  be  a  more  complex  problem.  Therefore,  centralized  control  is  preferred  in 
applications  for  smaller  fully  connected  formations. 

Platform-Based  Embedded  Software  Design  and  System  Integration  for 
Autonomous  Vehicles 

B.  Horowitz,  J.  Liebman,  C.  Ma,  T.  J.  Koo,  A.  Sangiovanni-Vincentelli, 
and  S.  Sastry  used  the  concepts  of  platform-based  design  to  develop  a 
methodology  for  the  design  of  automatic  control  systems  that  built  in  modularity 
and  correct-by-construction  procedures.  We  illustrated  our  strategy  by  describing 
the  application  of  the  methodology  to  the  design  of  a  time-based  control  system 
for  a  helicopter-based  Uninhabited  Aerial  Vehicle  (UAV)  so  that  the  following 
objectives  can  be  met: 

a.  The  use  of  platform-based  design  allows  us  to  build  a  bridge  between  the 
time-based  controller  application  and  the  non-time-based  sensors  and  actuators. 

b.  A  time-based  controller  eliminates  the  timing  irregularities  present  in  first 
generation  system.  Further,  the  Giotto  compiler  ensures  that  the  controller 
application  meets  its  timing  requirements. 

c.  Our  platform-based  design  achieves  a  high  degree  of  modularity.  For  example, 
to  substitute  a  different  sensor  suite  in  our  first  redesign  requires  only  changes  to 
the  data  processor  and  the  data  formatting  library.  The  data  processor  would 
require  a  different  sensor  initialization  routine  and  a  new  circular  buffer;  the 
formatting  library  would  need  a  new  format  conversion  routine.  However,  no  part 
of  the  controller  application  would  need  to  be  changed. 
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Chapter  Seven:  Experimentation 


Berkeley  Experimental  Activities 

We  have  a  rich  track  record  of  performing  successful  hardware  demonstrations 
on  our  Berkeley  Aerobotics  (BEAR)  platforms.  On  the  SEC  project:we  showed 

1 .  Vision  based  landing  (available  for  transition  partners) 

2.  Learning  techniques  for  complex  maneuvers  in  real  time 

3.  Giotto  based  embedded  software  design  (available  for  transition  partners) 

4.  UAV  Formation  Flight  in  String  Stable  configurations  available  for  use  in 
transition  role  for  night  flight  of  manned  helicopters  such  as  the  Apaches 
in  addition  to  UAV  operations. 

5.  UAV  Chicken  for  Conflict  Detection  and  Resolution 

As  we  have  shown  we  have  now  developed  some  very  light  weight  fixed  wing 
UAVs  (the  BAT)  and  some  electric  rotorcraft  for  use  in  cooperative  detection  of 
electromagnetic  radiation  or  for  cooperative  flight  path  planning  in  a  cluttered 
environment.  We  had  hardware  demonstrations  on  these  between  December 
2003  and  July  2004  as  per  the  following  schedule: 

1 .  Electric  rotorcraft  flight  in  October  2003 

2.  Navigation  of  electric  rotorcraft  UAV  in  urban  environment  including 
landing  December  2003. 

3.  Navigation  and  cooperative  and  non  cooperative  operations  of  rotorcraft 
UAVs  in  cluttered  urban  environments  May  2004 

Berkeley  Experiments  on  the  Boeing  Open  Experimental  Platform  July 
2004: 

Air  to  Air  Offensive  and  Defensive  Operations  for  UAVs 

The  aim  of  this  experiment  was  to  show  some  innovative  needs  for  UAVs  and 
UCAVs  in  future  operations  using  newly  developed  SEC  technology.  With  the 
use  of  multiple  UAVs  in  operations  (at  last  count  22  different  types  of  coalition 
UAVs  were  used  in  Operation  Iraqi  Freedom  OIF),  it  is  important  to  think  about 
capabilities  that  are  needed  to  perform  UAV  missions  typically  referred  to  as  DA3 
(dirty,  dull,  dangerous).  While  we  have  been  content  thus  far  with  Predators  and 
Global  Hawk’s  being  unarmed,  and  frequently  at  the  mercy  of  adversarial  air 
forces  (especially  in  a  pre-combat  phase),  it  is  important  to  enhance  the 
survivability  of  UAVs  by  endowing  them  with  the  ability  to  defend  themselves. 

This  is  perhaps  less  than  asking  for  UAVs  to  perform  Air  to  Air  combat  operations 
but  requires  thought  about  rapid  maneuvers  and  the  ability  to  evade  and  engage 
enemy  aircraft  which  will  frequently  have  a  speed  advantage  and  possibly 
maneuverability  and  G-force  limitations.  We  feel  the  need  for  this  capability  for 
both  strategic  and  tactical  assets,  but  our  capstone  demo  experiment  will  have  a 
tactical  conops  in  mind. 


39 


On  SEC  we  have  developed  numerous  techniques  for  model  predictive  control  of 
aircraft.  A  key  drawback  of  model  predictive  methods  thus  far  has  been  their 
inability  to  compute  and  refresh  with  a  high  refresh  rate  in  close  to  real  time.  We 
have  however  addressed  techniques  for  rapid  re-planning  of  model  predictive 
receding  horizon  schemes. 

In  the  three  demo  experiments  that  we  performed  we  highlighted  both  some  new 
technologies  and  some  new  capabilities  for  UAVs. 

Experiment  1:  UAV  as  Wingman 

The  aim  of  this  mission  was  to  have  the  UAV  follow  the  manually  flown  F-15E  as 
it  performs  a  sequence  of  maneuvers: 

•  Straight  and  Level  Flight 

•  Steady  Climb  from  one  altitude  to  another 

•  An  acrobatic  maneuver  such  as  a  barrel  roll  or  a  cobra  like  maneuver. 

In  this  experiment  we  used  the  T-33  as  the  UAV.  The  F-15E  was  the  manned 
leader.  Its  position  and  velocity  vector  was  communicated  to  the  UAV  flight 
controller  as  a  simulated  sensor  reading  by  the  UAV  of  the  F-15E  position  and 
intent.  The  F-15E  initially  started  at  a  fixed  altitude  20,000  feet  in  the  middle  of 
the  test  rangebfollow  a  straight  and  level  path  followed  by  a  climb  to  25,000  feet. 
At  the  end  of  the  climb  the  F-15E  turned  around  and  perform  several  repetitions 
of  an  acrobatic  maneuver  and  return  to  the  starting  point.  The  UAV  started  at  an 
altitude  of  25,000  feet  at  a  distance  of  3  nautical  miles  from  the  F-15E  at  the 
initial  time.  It’s  initial  goal  will  be  to  close  up  to  the  F-15E  with  an  offset  of 
1  nautical  mile  in  the  straight  and  level  path.  The  tracking  performance  of  the 
UAV  controller  was  evaluated  with  respect  to  a  1  n  mile  separation  requirement. 

Success  Metrics  used  were: 

•  Time  delay  in  forming  up  (entering  formation) 

•  Tracking  accuracy  in  the  climb  maneuver 

•  Success  metric  to  be  defined  for  the  acrobatic  maneuver  including 
spacing,  time  delay  and  overall  shape  of  the  UAV  maneuver. 


Experiment  2:  UAV  as  Evader 

The  UAV  is  given  a  speed  disadvantage  of  0.8  and  maneuverability  advantage 
of  more  allowable  Gs  by  constraining  the  capabilities  of  the  Pursuer  F-15E.  The 
UAV  is  asked  to  confine  its  maneuvers  in  a  certain  envelop  of  the  test  range  and 
is  given  the  task  of  keeping  the  Pursuer  F-15E  off  its  tail.  Keeping  the  F-15E  off 
its  tail  is  defined  as  keeping  the  F-15  out  of  a  cone  shaped  region  behind  the 
UAV  with  the  height  of  the  cone  being5  nautical  miles. 

Several  evasion  strategies  were  attempted  for  different  pursuer  strategies: 

•  Min-max 
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•  Stackelberg  strategy  based  on  a  model  of  the  pursuer’s  intent. 

The  UAV  and  F-15E  entered  the  airspace  at  different  altitudes  and  at  opposite 
ends  of  the  test  range.  The  goal  of  the  UAV  is  to  traverse  the  test  range 
diagonally  from  its  entry  point  to  its  exit  point  without  being  targeted  by  the  F-15E 
or  to  stay  alive  for  10  minutes. 

The  first  experiment  called  for  no  prior  information  about  the  F-15E  pilot  (the 
min-max  strategy),  the  second  experiment  will  call  for  a  prior  model  of  a  search 
strategy  being  adopted  by  the  pursuer  (for  example  to  traverse  the  test  range 
according  to  a  sweeping  pattern),  the  Stackelberg  strategy. 

The  success  metric  here  was  simple:  to  win  the  game.  This  involves  either 
staying  alive  for  10  minutes  or  to  traverse  the  test  region  from  corner  to  corner  at 
a  fixed  altitude.  The  reason  we  have  this  either/or  success  metric  is  to  prevent 
the  ability  of  the  F-15E  to  exclusively  guard  the  “exit”  point  of  the  UAV  from  the 
test  range. 


Experiment  3:  UAV  as  a  Pursuer 

The  roles  are  reversed  from  Experiment  2  but  the  speed  and  maneuverability 
advantage  remain  the  same  and  stay  with  the  manned  aircraft.  Once  again  the 
game  is  won  by  the  evader  if  it  successfully  traverses  the  test  range  or  stays 
alive. 

Conclusion 

The  overall  experiments  on  the  Boeing  Open  Control  Platform  were  a  huge 
success.  The  Boeing  test  pilot  frequently  lost  to  our  UAV  and  was  openly 
complimentary  of  the  performance  of  our  combat  maneuvers. 
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